The Challenge:

The City of New Haven, where 77 percent of students are eligible for free and reduced-price meals, has been trying to reduce violence and tackle the challenges facing high-risk young people. In 2014, the Mayor’s Office and the Youth Services Department set up a program called Youth Stat. Through this program, they collaborate with the Board of Education, Police Department, and over 20 community-based organizations to provide services for high-needs young people. Schools refer young people with failing grades, poor attendance, school discipline infractions, or involvement in the criminal justice systems to the program. These young people are then enrolled into Youth Stat – in some cases, through persistent follow up outside of school by outreach workers. Youth Stat regularly convenes Youth Department officials, school and Board of Education staff, including teachers, as well as representatives from local providers at each school to discuss individual cases, review difficulties participants face, and identify services that can help meet their needs. Services range from therapy for acute trauma to vocational training and can also involve securing resources, such as a mattress if a youth needs to stay with a relative when the immediate home is unsafe. While there are many anecdotal success stories, Youth Stat officials did not have a good sense of what happened to individuals after they were referred to providers – in fact, there have been isolated reports of referrals not happening promptly and youths falling through the cracks. Youth Stat officials also lacked data on the program’s performance to help flag potential areas for improving outcomes.

The Project:

The GPL collaborated with staff from Youth Stat, the Board of Education, and affiliated service providers to develop performance management tools to enable them to flag and troubleshoot operational challenges in real time to improve outcomes for high-risk youth. As part of this process, the GPL and New Haven:

1. Produced process and outcome metrics for two Youth Stat service providers, Project Youth Court and VETTS veteran mentoring. The process metrics, which are unique to each provider model, track the movement of youths through programs and will be used to present information on enrollment, advancement, and program completion to Youth Stat leadership in the future. The recommended outcome metrics, which include school discipline incidents, arrests, and sentencing, can be used across programs. By using the same or similar outcome metrics, the City can compare performance across similar programs and also monitor system-wide improvement of outcomes for young people.

2. Developed the Active Contract Management quantitative performance management tool and piloted it for Project Youth Court. Through Active Contract Management meetings, Youth Stat leadership and providers examine process and outcome metrics for programs to identify trends, challenges, and areas of improvement. Data are reviewed for the period prior to youths’ enrollment in a program, during program participation, as well as after exit from the program. In addition, metrics may be broken out by referral source and by age. Meeting attendees collaboratively use the data to brainstorm strategies for overcoming barriers and driving better performance. For example, it may become apparent during a review of performance data that a particular provider has positive impacts on school absenteeism but not on participants’ grades. In response, Youth Stat staff could consider what academic supports might complement the program.

3. Designed and piloted the qualitative performance management tool, named “Stories”. Youth Stat participants are randomly selected on a regular basis to be interviewed about their experience with Youth Stat and affiliated programs. During “Stories” meetings, Youth Stat leadership will review trends and individual experiences from these interviews. The goal is that the qualitative information will complement the analysis of performance data through Active Contract Management, allowing Youth Stat staff and service providers to uncover trends that the data may not highlight or to interpret trends revealed by the data. Attendees can discuss findings to identify challenges and develop adjustments to the program to drive performance. For example, it may become apparent during a review of performance data that wait times to begin a particular program are too long. Interviews with young people may indicate that this is due to confusion about how to enroll in the program. In response, the Youth Stat staff could clarify with the provider the expected service delivery timeline after a referral and who is responsible for onboarding program participants. In addition, success stories that may emerge from the interviews can also help the City communicate the potential of Youth Stat to change the lives of youth, which is critical as New Haven seeks greater political and financial support to expand the program to additional schools.

The Results:

New Haven is in the process of: 1) selecting a point person who will manage the collection of Active Contract Management and “Stories” data, lead the meetings to review data, and work with Youth Stat leadership to design strategies to improve outcomes; 2) building new reporting functions into the program’s case management software that would allow staff to track select outcomes for young people by provider and over time; and 3) deciding which requirements for data collection and reporting to include in new or renewed contracts with service providers that are affiliated with Youth Stat.