Implementation Examples: Publications

Below are summaries of articles on implementing specific practices rated at least a 3 (Promising Research Evidence) on the CEBC Scientific Rating Scale along with links to the complete article. Click on the citation to see the summary and link. Articles on general implementation issues can be found in the Tools & Resources section.

Adolescent Community Reinforcement Approach (A-CRA)

Hunter, S. B., Han, B., Slaughter, M. E., Godley, S. H., & Garner, B. R. (2015). Association between implementation characteristics and evidence-based practice sustainment: A study of the Adolescent Community Reinforcement Approach. Implementation Science, 10(173), 1–11.

    Article summary: This study examined common implementation influences on sustainability by following multiple community-based organizations that were funded by the Substance Abuse and Mental Health Services Administration (SAMHSA) to implement the Adolescent Community Reinforcement Approach (A-CRA). Data collection included key informant interviews and surveys. Results showed that sustainability was strongly related to the following factors: time since funding loss, implementation quality during funding period, agency focus, funding stability, political support, and staff perceptions of the treatment’s complexity and implementation difficulty.
    Link to article: doi:10.1186/s13012-015-0364-4

C.A.T. Project

Podell, J. L., Kendall, P. C., Gosch, E. A., Compton, S. N., March, J. S., Albano, A. M., ... Piacentini, J. C. (2013). Therapist factors and outcomes in CBT for anxiety in youth. Professional Psychology: Research and Practice, 44(2), 89-98.

    Article summary: Study of Coping Cat and the C.A.T. Project that showed therapists who were more collaborative and empathic, followed the treatment manual, and implemented it in a developmentally appropriate way had youth with better treatment outcomes. Therapist coach style was a significant predictor of child-reported outcome, with the collaborative coach style predicting fewer child-reported symptoms. Higher levels of therapist prior clinical experience and lower levels of prior anxiety-specific experience were significant predictors of better treatment outcome. Findings suggest that although all therapists used the same manual-guided treatment, therapist style, experience, and clinical skills were related to differences in child outcome.
    Link to article: doi:10.1037/a0031700

Coping Cat

Chu, B. C., Crocco, S. T., Arnold, C. C., Brown, R., Southam-Gerow, M. A., & Weisz, J. R. (2015). Sustained implementation of cognitive-behavioral therapy for youth anxiety and depression: Long-term effects of structured training and consultation on therapist practice in the field. Professional Psychology: Research and Practice, 46(1), 70-79.

    Article summary: This study interviewed 23 community-based therapists who had completed intensive training in cognitive-behavioral therapy for either anxiety or depression. Therapists were interviewed 3 to 5 years after completion of the initial training. Therapists viewed the training as a whole as effective; however, parts of each protocol were more frequently used than others (i.e., a majority of therapists used parts of Coping Cat and PASCET compared to therapists who used the whole protocol). Problem-solving and cognitive restructuring were used the most and exposure exercises and homework were used the least, respectively, for anxiety and depression. Results suggest that specific protocols perceived by therapists to be difficult to implement are less likely to be sustained.
    Link to article: doi:10.1037/a0038000

Podell, J. L., Kendall, P. C., Gosch, E. A., Compton, S. N., March, J. S., Albano, A. M., ... Piacentini, J. C. (2013). Therapist factors and outcomes in CBT for anxiety in youth. Professional Psychology: Research and Practice, 44(2), 89-98.

    Article summary: Study of Coping Cat and the C.A.T. Project that showed therapists who were more collaborative and empathic, followed the treatment manual, and implemented it in a developmentally appropriate way had youth with better treatment outcomes. Therapist coach style was a significant predictor of child-reported outcome, with the collaborative coach style predicting fewer child-reported symptoms. Higher levels of therapist prior clinical experience and lower levels of prior anxiety-specific experience were significant predictors of better treatment outcome. Findings suggest that although all therapists used the same manual-guided treatment, therapist style, experience, and clinical skills were related to differences in child outcome.
    Link to article: doi:10.1037/a0031700

Family Group Decision Making

Rauktis, M. E., McCarthy, S., Krackhardt, D., & Cahalane, H. (2010). Innovation in child welfare: The adoption and implementation of Family Group Decision Making in Pennsylvania. Children and Youth Services Review, 32(5), 732-739.

    Article summary: This paper explores what system factors influenced the adoption of Family Group Decision Making (FGDM) in Pennsylvania and what the perceived barriers and facilitators of adoption and sustainability are according to the individuals involved in FGDM. The findings reveal that maltreatment and poverty rates and the size of the population of children are non-significant predictors of a county using FGDM, but having a FGDM pilot grant is a significant predictor, along with having had a system of care initiative. Population density and number of caseworkers also are significant, if weaker, predictors; population density became insignificant once the two largest counties, Philadelphia and Allegheny were removed from the model. Having a neighboring county that practices FGDM e.g. the neighborhood effect is the most powerful predictor and contributes a sizeable effect. The individuals working in counties new to FGDM perceive that adoption required additional resources such as more staff, money and training. The survey results from counties that recently adopted FGDM compared to counties that have more experience with FGDM suggest that leadership becomes even more important in maintaining progress in FGDM implementation compared with leadership needed to begin FGDM. The findings suggest that to facilitate the adoption of FGDM, funders should look to strategically place new programs close to established programs, provide start-up funding and utilize networks established through system collaboration activities. Continued research in the nature of innovation in child welfare practice using theories and analyses more commonly associated with sociology and economics may better inform the child welfare systems change efforts.
    Link to article: doi:10.1016/j.childyouth.2010.01.010

Incredible Years

Webster-Stratton, C. H., Reid, M. J., & Marsenich, L. (2014). Improving therapist fidelity during implementation of evidence-based practices: Incredible Years program. Psychiatric Services, 65(6), 789-795.

    Article summary: Thus study compared two ways of training therapists to deliver the Incredible Years parenting program. Therapists were trained using either the standard three-day workshop model, based on active, experiential, self-reflective, principle-based learning, video modeling, and manuals, or an enhanced training model that included all elements of the workshop model plus ongoing expert coaching, video review of and feedback on group sessions, and consultation for therapists and agency supervisors. Therapists receiving the enhanced training model were significantly stronger in four of the domains: practical support, collaboration, knowledge, and skill at mediating vignettes. Consultation and expert coaching for training therapists beyond the standard three-day training enhanced skills and therapists adherence to the model.
    Link to article: doi:10.1176/appi.ps.201200177

KEEP (Keeping Foster and Kin Parents Supported and Trained)

Chamberlain, P., Feldman, S. W., Wulczyn, F., Saldana, L., & Forgatch, M. (2015). Implementation and evaluation of linked parenting models in a large urban child welfare system. Child Abuse & Neglect. Advance online publication.

    Article summary: This paper examined the implementation of two parenting interventions, KEEP (Keeping Foster and Kin Parents Supported and Trained) and Parenting Through Change for Reunification (PTC-R; a group-based form of the Parent Management Training Oregon Model), across five private agency pilot sites in order to decrease placement disruptions, length of stay in foster care and re-entry into foster care. A practice-based case supervision model, called R3, was also developed and implemented across all pilot sites to assist case workers with retaining evidence-based principles in everyday interactions with clients. Implementation barriers, opportunities, and client outcomes are presented and a strategy for creating a path to sustainability is discussed.
    Link to article: doi:10.1016/j.chiabu.2015.09.013

Saldana, L., & Chamberlain, P. (2013, April). Scaling up two evidence-based practices for children's mental health. CYF News, American Psychological Association.

Multidimensional Family Therapy (MDFT)

Rowe, C., Rigter, H., Henderson, C., Gantner, A., Mos, K., Nielsen, P., & Phan, O. (2013).Implementation fidelity of Multidimensional Family Therapy in an international trial. Journal of Substance Abuse Treatment, 44(4), 391-399.

    Article summary: This study reports on the implementation fidelity of MDFT across these five Western European sites in terms of treatment adherence, dose and program differentiation.
    Link to article: doi:10.1016/j.jsat.2012.08.225

Saldana, L., & Chamberlain, P. (2013, April). Scaling up two evidence-based practices for children's mental health. CYF News, American Psychological Association.

MultiSystemic Therapy (MST)

Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78(4), 537-550.

    Article summary: This article reports on the results of a randomized controlled trial using ARC (for Availability, Responsiveness and Continuity) organizational intervention to implement MST.
    Link to article: doi:10.1037/a0019160

Multi-Family Psychoeducational Psychotherapy (MF-PEP)

MacPherson, H. A., Fristad, M. A., & Leffler, J. M. (2014). Implementation of Multi-Family Psychoeducational Psychotherapy for childhood mood disorders in an outpatient community setting. Journal of Marital and Family Therapy, 40(2), 193-211.

    Article summary: Preliminary descriptive findings based on observations and self-report questionnaires demonstrated implementation outcomes of acceptability, adoption, appropriateness, feasibility, implementation cost, penetration, and sustainability of MF-PEP at two community clinics. Parents also demonstrated significant improvement in knowledge of mood disorders after treatment. Preliminary results support implementation of MF-PEP in practice settings and suggest community-based MF-PEP may be associated with improvement in clinical outcomes.
    Link to article: doi:10.1111/jmft.12013

Parent-Child Interaction Therapy (PCIT)

Herschell, A. D., Kolko, D. J., Scudder, A. T., Taber-Thomas, S., Schaffner, K. F. Hiegel. S. A., ... Mrozowski, S. (2015). Protocol for a statewide randomized controlled trial to compare three training models for implementing an evidence-based treatment. Implementation Science, 10(133), 1-16.

    Article summary: This article describes the protocol for examining the impact of three different training models (Learning Collaborative, Cascading Model, and Distance Education) on training outcomes, implementation outcomes for Parent-Child Interaction Therapy (PCIT), and client outcomes. Fifty clinicians were randomized into one of the three training conditions. The study is currently in progress with family enrollment expected to continue through December 31, 2016. Training variables to be examined include clinician knowledge, skills, and attitudes. Implementation variables to be examined include acceptability, adoption, appropriateness, feasibility, fidelity, penetration, sustainability, and cost across training conditions. Client variables to be examined include parent and child behavioral and mental health and parenting practices. Training and implementation measures are scheduled at baseline, 6, 12, and 24 months (1-year follow-up). Client measures are scheduled at pretreatment, 1, 6, and 12 months after pretreatment.
    Link to article: doi:10.1159/s13012-015-0324-z

Parent Management Training Oregon Model (PMTO)

Forgatch, M. S., & DeGarmo, D. S. (2011). Sustaining fidelity following the nationwide PMTO implementation in Norway. Prevention Science, 12, 235- 246.

    Article summary: This paper examined several issues related to fidelity in the nationwide implementation of Parent Management Training - Oregon Model (PMTO) in Norway, using multiple generations of trainers.
    Link to article: doi:10.1007/s11121-011-0225-6

Primary and Secondary Control Enhancement Training (PASCET)

Chu, B. C., Crocco, S. T., Arnold, C. C., Brown, R., Southam-Gerow, M. A., & Weisz, J. R. (2015). Sustained implementation of cognitive-behavioral therapy for youth anxiety and depression: Long-term effects of structured training and consultation on therapist practice in the field. Professional Psychology: Research and Practice, 46(1), 70-79.

    Article summary: This study interviewed 23 community-based therapists who had completed intensive training in cognitive-behavioral therapy for either anxiety or depression. Therapists were interviewed 3 to 5 years after completion of the initial training. Therapists viewed the training as a whole as effective; however, parts of each protocol were more frequently used than others (i.e., a majority of therapists used parts of Coping Cat and PASCET compared to therapists who used the whole protocol). Problem-solving and cognitive restructuring were used the most and exposure exercises and homework were used the least, respectively, for anxiety and depression. Results suggest that specific protocols perceived by therapists to be difficult to implement are less likely to be sustained.
    Link to article: doi:10.1037/a0038000

SafeCare

Hurlburt, M., Aarons, G. A., Fettes, D., Willging, C., Gunderson, L., & Chaffin, M. J. (2014). Interagency Collaborative Team model for capacity building to scale-up evidence-based practice. Children and Youth Services Review, 39, 160-168.

    Article summary: This paper describes the Interagency Collaborative Team (ICT) model and present preliminary qualitative results from the use of the implementation model in one large scale EBP implementation. Qualitative interviews were conducted to assess challenges in building system, organization, and home visitor collaboration and capacity to implement the EBP. Data collection and analysis centered on EBP implementation issues, as well as the experiences of home visitors under the ICT model. Six notable issues relating to implementation process emerged from participant interviews, including: (a) initial commitment and collaboration among stakeholders, (b) leadership, (c) communication, (d) practice fit with local context, (e) ongoing negotiation and problem solving, and (f) early successes.
    Link to article: doi:10.1016/j.childyouth.2013.10.005

Whitaker, D. J., Ryan, K. A., Wild, R. C., Self-Brown, S., Lutzker, J. R., Shanley, J. R., ... Hodges, A. E. (2012). Initial implementation indicators from a statewide rollout of SafeCare within a child welfare system. Child Maltreatment, 17(1), 96-101.

    Article summary: This paper describes the initial findings from a statewide effort to implement SafeCare® into a state child welfare system. A total of 50 agencies participated in training, with 295 individuals entering training to implement SafeCare. The quality of SafeCare uptake during training and implementation was high with trainees performing very well on training quizzes and role-plays, and demonstrating high fidelity when implementing SafeCare in the field (performing over 90% of expected behaviors). However, the quantity of implementation was generally low, with relatively few providers (only about 25%) implementing the model following workshop training.
    Link to article: doi:10.1177/1077559511430722

Aarons, G. A., Sommerfeld, D. H., Hecht, D. B., Silovsky, J. F., & Chaffin, M. J. (2009). The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. Journal of Consulting and Clinical Psychology, 77(2), 270-280.

    Article summary: Staff retention is an ongoing challenge in mental health and community-based service organizations. Little is known about the impact of practice implementation on the mental health and social service workforce. This study examined the effect of practice implementation and ongoing fidelity monitoring on staff retention in a children’s services system. Analyses revealed greater staff retention in the condition where the practice was implemented along with ongoing fidelity monitoring presented to staff as supportive consultation. These results should help to allay concerns about staff retention when implementing practices where there is good values-innovation fit and when fidelity monitoring is designed as an aid and support to service providers in providing a high standard of care for children and families.
    Link to article: http://dx.doi.org/10.1037%2Fa0013223

Aarons, G. A., Green, A. E., Palinkas, L. A., Self-Brown, S., Whitaker, D. J., Lutzker, J. R., ... & Chaffin, M. J. (2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7(32), 1-9.

    Article summary: This DAP article gives a practical application of EPIS as an implementation strategy including multilevel assessment during the exploration phase, and the use of implementation resource teams during preparation and implementation phases.
    Link to article: doi:10.1186/1748-5908-7-32

Willging, C. E., Green, A. E., Gunderson, L., Chaffin, M., & Aarons, G. A. (2014). From a perfect storm to smooth sailing: Policymaker perspectives on implementation and sustainment of an evidence-based practice in two states. Child Maltreatment, 20(1), 24-36.

    Article summary: Conducted multiple interviews with policymakers implementing SafeCare to identify the commonality, diversity, and complexity of implementation challenges; approaches to overcoming those challenges; and the importance of system-level contextual factors in ensuring successful implementation. Key findings centered on building support and leadership for EBPs; funding and contractual strategies; partnering with stakeholders; tackling challenges via proactive planning and problem solving; and the political, legal, and systemic pressures affecting EBP longevity.
    Link to article: doi:10.1177/1077559514547384

Teaching-Family Model

Fixsen, D., Blase, K., Timbers, G., & Wolf, M. (2007). In search of program implementation: 792 replications of the Teaching-Family Model. The Behavior Analyst Today, 8(1), 96-110.

    Article summary: This paper presents an analysis of 792 attempted replications of the Teaching-Family Model over a period of 15 years. The analysis reveals certain aspects of the treatment program that were found to be sufficient conditions for treatment program implementation and survival, including program fit with local agency structures, the need for on-going training and coaching, and the requirements for staff selection and support.
    Link to article: http://eric.ed.gov/?id=EJ800975

Triple P - Positive Parenting Program® - Level 4 (Level 4 Triple P)

Asgary-Eden, V., & Lee, C. M. (2012). Implementing an evidence-based parenting program in community agencies: What helps and what gets in the way? Administration and Policy in Mental Health and Mental Health Services Research, 39(6), 478-488.

    Article summary: Adoption of manualized parenting programs for families by community agencies requires an understanding of variables that influence implementation. Managers and service providers from 64 community agencies reported on variables that affected the implementation of Triple P, a manualized parenting program. Both types of stakeholders reported adequate office resources; over half the managers and over two-thirds of service providers reported adequate training. Adequate office resources and positive agency characteristics, including organizational climate, were associated with higher program usage. Service providers reports of the variables impacted their individual adherence rates; managers had broader perspectives of the quality of implementation in their organizations.
    Link to article: doi:10.1007/s10488-011-0371-y

Triple P - Positive Parenting Program® System (System Triple P)

Sanders, M. R., Prinz, R. J., & Shapiro, C. J. (2009). Predicting utilization of evidence-based parenting interventions with organizational, service-provider and client variables. Administration and Policy in Mental Health and Mental Health Services Research, 36(2), 133-143.

    Article summary: Multidisciplinary service providers (N = 611) who underwent training in the Triple P-Positive Parenting Program participated in a structured interview 6 months following training to determine their level of post-training program use and to identify any facilitators and barriers to program use. Findings revealed that practitioners who had received training in Group Triple P, received positive client feedback, had experienced only minor barriers to implementation, and had consulted with other Triple P practitioners following training were more likely to become high users of the program. Practitioners were less likely to use the program when they had lower levels of confidence in delivering Triple P and in consulting with parents in general, had difficulties in incorporating Triple P into their work, and where there was low workplace support. These findings highlight the importance of considering the broader post training work environment of service providers as a determinant of subsequent program use.
    Link to article: doi:10.1007/s10488-009-0205-3

Shapiro, C. J., Prinz, R. J., & Sanders, M. R. (2012). Facilitators and barriers to implementation of an evidence-based parenting intervention to prevent child maltreatment: The Triple P-Positive Parenting Program. Child Maltreatment, 17(1), 86-95.

    Article summary: This study involved structured interviews with service providers who had been trained previously in the delivery of the Triple P-Positive Parenting Program. These follow-up interviews, conducted an average of about 2 years after professional in-service training, provided the basis for examining predictors of sustained program use. Several provider and organization-level characteristics significantly predicted program use, including provider self-confidence after training, fit of program with ongoing duties, availability of post training support, and perceived benefit of intervention for children and families.
    Link to article: doi:10.1177/1077559511424774

Wraparound

Bruns, E. J., Burchard, J. D., Suter, J. C., Leverentz-Brady, K., & Force, M. M. (2004). Assessing fidelity to a community-based treatment for youth: The Wraparound Fidelity Index. Journal of Emotional and Behavioral Disorders, 12, 79 - 89.

    Article summary: Describes the Wraparound Fidelity Index (WFI), a multi-informant measure designed to assess providers' adherence during service delivery to the essential elements of Wraparound.
    Link to article: doi: 10.1037/a0013223