The Challenge:

Arizona’s Department of Child Safety (DCS) faced a troubling trend—families who had been in the department’s care often experienced subsequent incidents of child maltreatment despite having received services designed to prevent future incidents of abuse and neglect. From 2016-2018, DCS had successfully reduced the number of children living in out-of-home settings by 25%, and increased the use of family support services— including parenting skills training, referrals to other resources, and case coordination— designed to improve family stability and outcomes. Despite Arizona’s comparatively low rates of recurring maltreatment overall, the department discovered that family support services did not appear to be sufficiently preventing repeated incidents of maltreatment. For example, among families who received supportive in-home preservation services, nearly 1 in 5 were re-reported for future maltreatment, with the majority of these re-reports resulting in the removal of at least one child from the home.

In order to improve outcomes for families receiving in-home services, DCS set out to improve the quality of its contracted in-home preservation services. To identify the underlying causes of poor service delivery and generate and test solutions, DCS established collaborative practices between service providers and agency staff, connected previously siloed data to drive insights into client outcomes, and established mechanisms for sustaining improvements to services.

The Project:

DCS led an agency-wide strategic effort to standardize and improve the quality of in-home preservation services. Supported by GPL technical assistance, project partners:

1. Rolled out collaborative performance management practices that use provider and agency data to identify program challenges and implement improvements to services

Project partners rolled out an active contract management (ACM) strategy, consisting of high-frequency, data-informed meetings between DCS and the provider community designed to identify and address barriers to improving service delivery. As part of this process, department staff systematically identify key outcome and performance measures, monitor progress, and test hypotheses on specific trends using new approaches to matching child-level data sourced from service providers with individual outcome and demographic information from the state’s case management system. For example, data reviews revealed inconsistent understanding of the Caregiver Protective Capacities (CPC) framework, a framework to help identify a parent’s or caregiver’s ability to care for and keep a child safe. Poor understanding of the CPCs prevented providers from systematically assessing issues, prioritizing the interventions, and setting goals for improvements to caregiver behavioral, emotional, and cognitive capacities. In response to this finding, the department is conducting sessions to clearly define the capacities and revamping its training and additional resources for providers to assess, prioritize, and enhance the capacities for families receiving services.

To improve its ability to work effectively with providers, DCS actively engaged providers in collaborative sessions to diagnose issues, test out potential solutions, and share leading practices. DCS evidenced the shift from oversight to collaboration by demonstrating that responsibility for addressing shortfalls in performance would be shared by the department. For example, regular case reviews with providers revealed a lack of communication with DCS caseworkers during the development of service plans for children and families. This led to collaboration sessions to issue new guidance and tools, including program-wide standards and templates for both providers and caseworkers to facilitate joint production of future service plans.

2. Reengineered data collection to facilitate better identification of program challenges and evaluation of solutions

The high frequency, collaborative meetings between DCS and providers required the use of better quality data that would allow participants to more accurately pinpoint areas where services could be improved. Previously, data had been siloed between the two parties, with DCS collecting information on outcomes such as removals and repeat maltreatment, and providers reporting only summary-level data on child- and family-level treatment information. DCS worked with providers to select a core set of service provision and outcome metrics, improve the quality of reported data, broaden the type of information being gathered to include more qualitative, field-level insights, and combine and share data between the department and the providers through ACM. As part of this work, DCS overhauled its oversight processes, provider site visit tools and protocols troubleshoot issues, randomized case file pulls to assess quality of service delivery, and the implementation of routine data collection, aggregation, and visualization procedures.

3. Established a new DCS team tasked with driving performance improvements of child and family service contracts

DCS established a new unit to sustain the improvements it had made to its practice. The Fidelity and Compliance Services (FCS) Unit oversees service delivery and performance monitoring for child and family service contracts. Staffed by program-specific service coordinators, the unit is responsible for proactively managing performance and delivery of services and serving as a resource for field staff. Additionally, the new team provides data-related resources and tailored coaching, including the creation and maintenance of replicable data flows, development of data visualization methods, and evaluation techniques. These tools help staff to assess service quality (including timeliness, treatment dosage, and completion rates) and track outcomes (including enhanced family capacity and lack of subsequent maltreatment).

The Results:

These initiatives contributed to improved outcomes for families, formed stronger relationships between the agency and the service provider community, and created a model of sustainable performance management. DCS:

1. Cut reports of maltreatment and removals of children for families receiving in-home services in half

The share of families receiving in-home services who are implicated in subsequent re-reports of abuse or neglect and child removal during service delivery has steadily decreased, with approximately 40% fewer families re-reported and 50% fewer families experiencing removal(s) from peaks 6 months prior to implementing the new performance management practices. Now equipped with data on key outcome and service delivery measures, providers are better able to target efforts to improve services, such as increasing family engagement and prioritizing services based on family need. Improvements in service delivery have allowed in-home services to better preserve family unity and minimize subsequent intervention.

2. New collaborative relationships with the provider community

DCS’s interactions with providers no longer focus predominantly on compliance and logistical topics, but on improving service delivery and outcomes for children and families. Providers are explicitly encouraged and supported by DCS in their efforts to innovate and attempt new service delivery strategies within the scope of the contract. For example, pilot programs are currently being designed to more quickly enroll and retain families in supervised visitation programs. In addition, DCS has strengthened the feedback loop between service providers, case workers, and contract administration to systematically collect information across the provider cohort and rapidly respond when patterns of issues emerge during the contract period.

3. Sustainable, government-led capacity to identify and address barriers to high quality service delivery

Through the creation of a new unit, DCS staff are able to better target intensive technical assistance to specific providers or uncover systemic challenges in service delivery or outcomes. In addition, DCS has expanded ACM techniques to all child- and family-facing contracts, including in-home services, home visiting, parent education, supervised visitation, foster family recruitment and licensing, and congregate care.