Aarons, G. A., Green, A. E., Trott, E., Willging, C. E., Torres, E. M., Ehrhart, M. G. & Roesch, S. C. (2016). The roles of system and organization leadership in system-wide evidence-based intervention sustainment: A mixed-method study. Administration and Policy in Mental Health and Mental Health Services Research, 43, 991-1008.

Article Summary: Using the Exploration, Preparation, Implementation, Sustainment (EPIS) framework, this study examines the influence of leadership in the outer (systems) and inner (teams) settings on the sustainment of a new EBP. Leadership in both the inner and outer settings predicted future sustainment of the EBP. In the outer setting, sustainment was associated with leaders championing the practice, marketing practice to stakeholders, taking action to institutionalize the project with funding, contracting, and system improvement plans, and fostering on-going collaboration between stakeholders. In the inner setting, important leadership activities included championing the new practice and providing support to service providers.
Link to article: doi:10.1007/s10488-016-0751-4

Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4-23.

Article Summary: This paper presents EPIS (Exploration, Preparation, Implementation, Sustainment), a multi-level, four phase model of the implementation process which is derived from the research literature, and applies it to public sector services, such as child welfare and mental health. Features of the model that are likely to be particularly important in each phase are highlighted while considering the internal and external contexts common to public sector service systems.
Link to article: doi:10.1007/s10488-010-0327-7

Aarons, G. A., & Palinkas, L. A. (2007). Implementation of evidence-based practice in child welfare: Service provider perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 34(4), 411-419.

Article Summary: This study examined the factors that were critical in the implementation of evidence-based practices (EBP) in child welfare through interviews with case managers engaged in implementing an EBP to reduce child neglect. Six primary factors were identified as critical determinants of EBP implementation: (1) Acceptability of the EBP to the caseworker and to the family, (2) Suitability of the EBP to the needs of the family, (3) Caseworker motivations for using the EBP, (4) Experiences with being trained in the EBP, (5) Extent of organizational support for EBP implementation, and (6) Impact of EBP on process and outcome of services. These factors reflect two broader themes of attitudes toward or assessments of the EBP itself and experiences with learning and delivering the EBP. This information can be used by agencies as they prepare to implement a new practice or change.
Link to article: doi:10.1007/s10488-007-0121-3

Aarons, G. A., Sommerfeld, D. H., Hecht, B., Silovsky, J. F., & Chaffin, M. (2009). The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. Journal of Consulting and Clinical Psychology, 77(2), 270.

Article Summary: Staff retention is an ongoing challenge in mental health and community-based service organizations. Little is known about the impact of evidence-based practice (EBP) implementation on the mental health and social service workforce. This study examined the effect of EBP implementation and ongoing fidelity monitoring on staff retention in a children's services system. Analyses revealed greater staff retention when the EBP was implemented along with ongoing fidelity monitoring presented to staff as supportive consultation. These results should help to allay concerns about staff retention when implementing EBPs that are a good fit with agency and community needs. In addition, designing fidelity monitoring as an aid and support to service providers in providing a high standard of care for children and families, rather than as a potentially punitive process to identify deficiencies, also bolsters retention.
Link to article: doi:10.1037/a0013223

Beidas, R. S., Marcus, S., Aarons, G. A., Hoagwood K. E., Schoenwald, S., Evans, A. C., Hurford, M. O., Hadley, T., Barg, F. K., Walsh, L. M., Adams, D. R., & Mandell, D. S. (2015). Predictors of community therapists' use of therapy techniques in a large public mental health system. JAMA Pediatrics. 169(4), 374-382. https://doi.org/10.1001/jamapediatrics.2014.3736

Article Summary: This study examined the relative contribution of individual and organizational factors on therapist self-reported use of cognitive-behavioral, family, and psychodynamic therapy techniques within the context of a large-scale effort to increase use of evidence-based practices in an urban public mental health system serving youth and families. Older therapists and therapists with more open attitudes were more likely to endorse use of cognitive-behavioral therapy techniques, as were those in organizations that had spent fewer years participating in evidence-based practice initiatives, had more resistant cultures, and had more functional climates. Women were more likely to endorse use of family therapy techniques, as were those in organizations employing more fee-for-service staff and with more stressful climates. Therapists with more divergent attitudes and less knowledge about evidence-based practices were more likely to use psychodynamic therapy techniques.
Link to article: doi:10.1001/jamapediatrics.2014.3736

Bryson, S. A., Akin, B. A., Blase, K. A., McDonald, T., & Walker, S. (2014). Selecting an EBP to reduce long-term foster care: Lessons from a university-child welfare agency partnership. Journal of Evidence-Based Social Work, 11,208-221.

Article Summary: This article describes the EBP selection process and lessons learned between a state university, a public child welfare agency, and four private foster care providers in order to reduce long-term foster care. Key selection process activities included refining the target population and examining barriers to permanency, examining available empirical evidence, and selecting an established EBP developer or synthesizing common elements of EBPs. Lessons learned are divided into facilitators and barriers to success. Facilitators include the benefits of privatization, early involvement in all parties in proposal preparation, decision-making by consensus, equal distribution of resources by region, administration by a neutral third party (university), regular involvement of agency leaders with sufficient decision-making authority, core steering committee members included in federal grantee meetings, the state public child welfare agency representative present at steering committee, and the importance of identifying non-negotiable core elements of an EBP and their fit with the desired intervention point & population. Barriers included the contract bidding process leading to new partners and how to spread resources to providers across regions to better meet the challenges of implementing across rural areas.
Link to article: doi:10.1080/15433714.2013.850325

Glisson, C., & Green, P. (2011). Organizational climate, services, and outcomes in child welfare systems. Child Abuse & Neglect, 35(8), 582-591.

Article Summary: This study examines the association of organizational climate, casework services, and youth outcomes in child welfare systems over a 7-year follow-up period. Results from analyses of a nationwide sample of child welfare agencies and cases showed that maltreated youth served by child welfare systems with more engaged organizational climates have significantly better outcomes. This relationship was not affected by the quantity and quality of casework services.
Link to article: doi:10.1016/j.chiabu.2011.04.009

Glisson, C., Green, P., & Williams, N. J. (2012). Assessing the organizational social context (OSC) of child welfare systems: Implications for research and practice. Child Abuse & Neglect, 36(9), 621-632.

Article Summary: This study examines the association of child welfare system-level organizational culture and climate profiles with individual caseworker-level job satisfaction and organizational commitment. Caseworkers in the child welfare systems with the best organizational culture and climate profiles report higher levels of job satisfaction and organizational commitment. Organizational climates characterized by high engagement and functionality, and organizational cultures characterized by low rigidity are associated with the most positive work attitudes.
Link to article: doi:10.1016/j.chiabu.2012.06.002

Hanson, R. F., Self-Brown, S., Rostad, W. T., & Jackson, M. C. (2016). The what, when, and why of implementation frameworks for evidence-based practices in child welfare and child mental health service systems. Child Abuse & Neglect, 53,51-63.

Article Summary: This article summarizes the scientific literature on ten different implementation frameworks that are the most relevant to child welfare and child mental health settings, with an emphasis on the application of each framework toward services and policies for children and families impacted by abuse or trauma. The article discusses some considerations for framework selection, including the type of problem(s) being targeted, the scope of the implementation, the opportunity for trouble-shooting and strategy development, the framework's ability to consider inner and outer contexts, and the framewor's ability to provide structure and low costs associated to the widespread dissemination of implementation supports throughout the organization/system. Available frameworks are limited in attention paid to implementation efforts that target multidisciplinary providers. An innovative approach, called the Community-Based Learning Collaborative, is presented to address this limitation.
Link to article: doi:10.1016/j.chiabu.2015.09.014

Horwitz, S. M., Chamberlain, P., Landsverk, J., & Mullican, C. (2010). Improving the mental health of children in child welfare through implementation evidence-based parenting interventions. Administration and Policy in Mental Health and Mental Health Services Research, 37(1-2), 27-39.

Article Summary: This paper reviews what is known about efficacious parent-focused interventions that can improve the lives of children in Child Welfare and explores possible reasons why such interventions are rarely used by Child Welfare agencies. Data from interviews with six child welfare agencies suggest key features for increasing the implementation of efficacious practices to improve children's mental health. Most agencies identified staff resistance due to fear of the unknown/change or job loss as a major barrier to practice change. The most important drivers for exploring and adopting evidence-based practices (EBPs) were largely socio-political, such as funding, state/federal mandates and interest of leaders in improving practices. Financial benefits and staff interest were also identified as very important by programs that had explored practice change. EBPs that were considered most likely to be adopted were those that were consistent with the agency's current practices and philosophy and those that were most likely to benefit clients. Few agencies mentioned that being evidence based had influenced whether a new practice was adopted.
Link to article: doi:10.1007/s10488-010-0274-3

Horwitz, S. M., Hurlburt, M. S., Goldhaber-Fiebert, J. D., Palinkas, L. A., Rolls-Reutz, J., Zhang, ... Landsverk, J. (2014). Exploration and adoption of evidence-based practice by US child welfare agencies. Children and Youth Services Review, 39, 147-152.

Article Summary: This study examined the extent to which child welfare agencies adopt new practices and identified the barriers to and facilitators of adoption of new practices. A national survey of child welfare agencies showed that almost all agencies had started a new program or practice in the past 5 years but only a quarter were evidence-based and the main strategies used to explore new programs or practices were contacts with state or local colleagues. Factors that were associated with program success included internal support for the new practice and an existing evidence base for the practice.
Link to article: doi:10.1016/j.childyouth.2013.10.004

Lau, A. S. & Brookman-Frazee, L. (2016). The 4KEEPS study: Identifying predictors of sustainment of multiple practices fiscally mandated in children's mental health services. Implementation Science, 11, 1-8.

Article Summary: The Knowledge Exchange on Evidence-Based Practice Sustainment (4KEEPS) study is currently being conducted to examine the sustainment of evidence-based practices implemented within a fiscal mandate in the Los Angeles public mental health system. This article describes the 4KEEPS method for characterizing implementation sustainment and identifying the organizational and therapist characteristics that predict sustainment. Data sources being collected include archived organizational reports and documents, administrative claims data, and administering surveys to therapists and program managers featuring sustainment measures. The findings from this study will inform the development of implementation interventions to promote sustained delivery of evidence-based practices.
Link to article: doi:10.1186/s13012-016-0388-4

Lewis, C. C., Fischer, S., Weiner, B., Stanick, C., Kim, M., & Martinez, R. G. (2015). Outcomes for implementation science: An enhanced systematic review of instruments using evidence-based rating criteria. Implementation Science, 10(155), 1-17.

Article Summary: This article reviews available quantitative instruments that measure implementation outcomes relevant to mental or behavioral health settings. Measures include acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability. A total of 104 instruments were identified with only one instrument meeting the minimal criteria for evidence-based assessment. The article concludes that instruments measuring implementation outcomes are limited in availability and in psychometric quality.
Link to article: doi: 10.1159/s13012-015-0342-x

Mildon, R., & Shlonsky, A. (2011). Bridge over troubled water: Using implementation science to facilitate effective services in child welfare. Child Abuse & Neglect, 35(9), 753-756.

Article Summary: To maximize benefits to children and their families, evidence based practices (EBPs) need to be used competently in child welfare settings. However, limited attention has been paid to what is needed to implement EBPs in a range of real-world settings. Without proper implementation, an EBP can be made ineffective in real world application.
Link to article: doi:10.1016/j.chiabu.2011.07.001

Palinkas, L. A., Holloway, I. W., Rice, E., Fuentes, D., Wu, Q., & Chamberlain, P. (2011). Social networks and implementation of evidence-based practices in public youth-serving systems: A mixed-methods study. Implementation Science, 6(113), 1-11.

Article Summary: This study examines the structure and operation of social networks of information and advice and their role in making decisions as to whether to adopt new evidence-based practices (EBPs) among system leaders in 12 counties participating in a large randomized controlled trial. Systems leaders develop and maintain networks of information and advice based on roles, responsibility, geography, and friendship ties. Networks expose leaders to information about EBPs and opportunities to adopt EBPs; they also influence decisions to adopt EBPs. Individuals in counties at the same stage of implementation accounted for 83% of all network ties. Networks in counties that decided not to implement a specific EBP had no extra-county ties. Implementation of EBPs at the two-year follow-up was associated with the size of county, urban versus rural counties, and in-degree centrality. Collaboration was viewed as critical to implementing EBPs, especially in small, rural counties where agencies have limited resources on their own. Successful implementation of EBPs requires consideration and utilization of existing social networks of high-status systems leaders that often cut across service organizations and their geographic jurisdictions.
Link to article: doi:10.1186/1748-5908-6-113

Palinkas, L. A., Fuentes, D., Finno, M., Garcia, A. R., Holloway, I. W., & Chamberlain, P. (2014). Inter-organizational collaboration in the implementation of evidence-based practices among public agencies serving abused and neglected youth. Administration and Policy in Mental Health and Mental Health Services Research, 41(1), 74-85.

Article Summary: This study examined the role of inter-organizational collaboration in implementing new evidence-based practices (EBPs) for addressing problem behaviors in at-risk youth through interviews with systems leaders in probation, mental health, and child welfare departments. Three sets of collaboration characteristics were identified: (1) characteristics of collaboration process, (2) characteristics of the external environment, and (3) characteristics of participating organizations and individuals. Inter-organizational collaboration enables an exchange of information and advice and a pooling of resources individual agencies may require for successful implementation.
Link to article: doi:10.1007/s10488-012-0437-5

Peterson, A. E., Bond, G. R., Drake, R., McHugo, J., Jones, M., & Williams, J. (2014). Predicting the long-term sustainability of evidence-based practices in mental health care: An 8-year longitudinal analysis. The Journal of Behavioral Health Services & Research, 41(3), 337-346.

Article Summary: This study examined assessments of five evidence-based practices (EBPs) implemented in 49 sites in eight states at baseline and years 2, 4, and 8. Program characteristics, implementation characteristics, reinforcement activities, and sustainability factors were used to predict program survival status. The majority of predictors were not significant. At year 8, program sustainability was not related to any baseline characteristics. Similarly, fidelity and quality improvement, rated at year 2, did not predict sustainability at year 8. Supervisor turnover in year 4 predicted survival status in year 8, as discontinued programs rated leadership/supervisor turnover as a more negative influence on sustainability than programs that sustained.
Link to article: doi:10.1007/s11414-013-9347-x

Raghavan, R., Bright, C. L., & Shadoin, A. L. (2008). Toward a policy ecology of implementation evidence-based practices in public mental health settings. Implementation Science, 3(26), 1-9.

Article Summary: Mental health policymaking to support the implementation of evidence-based practices (EBPs) largely has been directed toward clinicians. However, implementation is known to be dependent upon a broader system of service delivery. Therefore, focusing exclusively on individual clinicians as targets of implementation is unlikely to result in sustainable and widespread implementation of EBPs. Policymaking that is informed by the implementation literature requires that policymakers deploy strategies across multiple levels, such as the organizational, regulatory, political, and social levels. The implementation literature suggests that a single policy decision, such as mandating a specific EBP, is unlikely to result in sustainable implementation. Policymaking that addresses implementation in an integrated way at the levels of provider organizations, governmental regulatory agencies, and their surrounding political and societal milieu is required to successfully and sustainably implement EBPs over the long term.
Link to article: doi:10.1186/1748-5908-3-26

Selden, S. C., & Sowa, J. E. (2015). Voluntary turnover in nonprofit human service organizations: The impact of high performance work practices. Human Service Organizations: Management, Leadership & Governance, 39(3), 182-207.

Article Summary: This study examines the relationship between the implementation of high performance work practices (HPWPS), an organizational framework of human resource and management practices, and voluntary staff turnover in human service organizations. Results showed that certain HPWPs, including onboarding, leadership development, compensation, and employee relations, are associated with lower voluntary turnover.
Link to article: doi:10.1080/23303131.2015.1031416

Williams, V., Deane, F. P., Oades, L. G., Crowe, T. P. Ciarrochi, J., & Andresen, R. (2016). A cluster-randomised controlled trial of values-based training to promote autonomously held recovery values in mental health workers. Implementation Science, 11, 1-11.

Article Summary: This study examined an intervention to increase mental health workers' feelings of autonomy to promote the sustainment of behavioral change after participating in employee skill development. One hundred forty-six mental health workers were randomly assigned to a structured values clarification group, where personal values and workplace values are both explored and prioritized as an additional component to an evidence-based staff development training program (called the Collaborative Recovery Model Training), or a control group that focused on identifying and problem-solving organizational barriers to the implementation of the newly acquired skills and practices. Study participants completed measures on autonomous motivation and the degree they plan to purposefully implement the new skills and practices. Results showed the structured values clarification group was useful in promoting integrated motivation for the changed practice and resulted in increased implementation planning greater than the control group.
Link to article: doi:10.1186/s13012-015-0363-5

Wilson, A. B., & Farkas, K. (2014). Collaborative adaptations in social work intervention research in real-world settings: Lessons learned from the field. Journal of Evidence-Based Social Work, 11, 183-192.

Article Summary: The article describes a collaborative approach between researchers and service practitioners to integrating and adapting two service models intended to treat young juvenile offenders with serious mental illness and substance use disorder and provide community reentry services with the goal of reducing recidivism. A shared decision-making model was used to facilitate the systematic decision making on what adaptations are required to support successful implementation of the intervention. Lessons learned include (1) ensuring clear explication at the beginning of the collaboration on what decisions are and are not shared and determining who is responsible for which decisions, (2) the importance of principal researchers holding strong backgrounds in both research and practice, including experience working in criminal justice settings, with the target population, and intervention modalities, (3) having mechanisms to handle referrals made by individuals who are outside of the project, (4) the importance of practitioners ability to identify future implementation problems, and (5) how the scarcity of resources in social services contributed to seasoned practitioners ability and willingness to commit to the one year project. Limitations include the lack of practitioner perspectives and findings from early implementation phases only.
Link to article: doi:10.1080/15433714.2013.847267

Wingood, G. M., & DiClemente, R. J. (2008). The adapt-itt model: A novel method of adapting evidence-based HIV interventions. JAIDS Journal of Acquired Immune Deficiency Syndromes, 47, S40-S46.

Article Summary: Few models exist to guide the adaptation of evidence-based practices (EBPs). This paper presents the ADAPT-ITT model, which consists of 8 sequential phases that developers and researchers can use as a method for adapting EBPs in a systematic fashion.
Link to article: doi:10.1097/QAI.0b013e3181605df1