This tool has received the Measurement Tools Rating of
A – Psychometrics Well-Demonstrated based on the published, peer-reviewed research available. The tool must have 2 or more published, peer-reviewed studies that have established the measure’s psychometrics (e.g., reliability and validity, sensitivity and specificity, etc.). Please see the
Measurement Tools Rating Scale for more information.
View
All Research Articles
-
Baxter, E. E., Alexander, P. C., Kraus, D. R., Bentley, J. H., Boswell, J. F., & Castonguay, L. G. (2016). Concurrent validation of the Treatment Outcome Package (TOP) for children and adolescents. Journal of Child and Family Studies, 25, 2415–2422. https://doi.org/10.1007/s10826-016-0419-4
Participants: 203 children and adolescents, ages 3–18 years, from a community sample
Sample / Population:
- Race/Ethnicity — Not specified
Summary:
The primary purpose of this study was to examine the concurrent validity of the Adolescent version of The Treatment Outcome Package (TOP) with the Child Behavior Checklist (CBCL) and the Strengths and Difficulties Questionnaire (SDQ) with a community sample. Results demonstrated that the TOP for adolescents has strong concurrent validity and measures constructs that are similar to those measured by well-established and widely used psychological assessments. The Adolescent TOP ADHD scale was highly correlated with both the SDQ Hyperactivity scale and the CBCL Attention Problems scale. The Adolescent TOP Depression scale correlated significantly with the SDQ Emotional Problems scale and both of the CBCL Depression scales (Anxious/Depressed and Withdrawn/Depressed). Finally, the TOP Violence scale was highly correlated with the SDQ Conduct scale, the CBCL Rule-breaking scale, and the CBCL Aggressive Behavior scale. Each of these correlations exceeded the Bonferroni-adjusted significance level.
-
Trudeau, K. J., Yang, J., Di, J., Lu, Y., & Kraus, D. R. (2023). Predicting successful placements for youth in child welfare with machine learning. Children and Youth Services Review, 153, 1–8. https://doi.org/10.1016/j.childyouth.2023.107117
Participants: 2,806 youth-aged behavioral health clients.
Sample / Population:
- Race/Ethnicity — 53% White, 26% Black/African American, 9% Unreported, 7% Hispanic/Latino, 4% Native American, 1% Asian, and <1% Mixed Race
Summary:
The objective of this study was to test the feasibility of using big data and machine learning techniques to help counties and states predict each child’s likelihood of success in high end, congregate care (i.e., residential treatment program), compared to lower-cost, wraparound services like outpatient therapy, school-based therapy, etc. The study used de-identified data that was collected in the course of clinical treatment for youth-aged behavioral health clients. Placement data with placement name, placement start date, and placement end date for each child per treatment episode was obtained. TOP data and placement data for clients were merged in one de-identified database for analysis. The algorithm was trained using a de-identified database of 1.4+ million administrations. Treatment episodes were divided into two groups: Psychiatric Residential Treatment Facility (i.e., “ever-PRTF” –most severe level-of-care) or non-PRTF treatment (i.e., “non-PRTF”) using recent demographic and clinical data. 2 × 2 Chi squares with Yates correction were conducted to test the hypothesis that the ratio of clients with good/bad outcomes was higher for the treatment episodes that matched the treatment recommendation based on the likelihood. The percentage of clients who experienced improved outcomes in the placement setting with the highest likelihood of success was calculated for each model (per domain; Total Score; Aggregate Score); 70% was the pre-stated minimum feasibility criterion. Multiple machine learning models achieved the a priori feasibility criteria of .70 or higher metrics (Area Under the Receiving Curve; AUROC). In this study, success was not only defined as making statistically significant progress in care. Success was further defined as a child achieving above-average risk-adjusted outcomes. When children were placed in a level of care that the model would not have recommended, the chance of above average success statistics did not change much (the good outcome to bad outcome ratio for TOP Total Score was 1.25). However, when the child was placed in the setting concordant with the model’s recommendation the number of children who had above average outcomes were four times higher than those that had below average outcomes.
