 Evaluation
of High-Risk Substance Abuse Prevention Initiatives
Macro developed and implemented a 5-year longitudinal quasi-experimental
design to test the effects of strategies to prevent or reduce the
use of alcohol, tobacco, and illicit drugs among youth at high risk
for these behaviors. The study examined 48 community-based programs
funded by the Center for Substance Abuse Prevention to demonstrate
an assortment of prevention approaches.
We collected data on outcomes from approximately 6,000 program
youth and 4,000 comparison youth at four points in time (program
entry, program exit, 6 months after exit, and 18 months after exit).
To help interpret client outcomes, we collected process and program
implementation data in annual visits to each site. We used a standardized
youth survey instrument to measure substance use, risk and resiliency
factors, and client characteristics and circumstances. To avoid
the “black box” problem whereby an evaluation finds
no effect because people received different services in different
doses, the design provided for measurement of services received
by individual participants. We standardized a site visit protocol
to sort interview information into categories that could easily
be merged with survey and dosage data.
In 2000, the American Evaluation Association gave this study an
award for best exemplifying the Association’s Guiding Principles
for Evaluation.
 Continuum of Care Evaluation
Macro is working with Connecticut’s Department of Children and
Families to test ways of caring for children with severe emotional disturbance
who are placed in a residential or group homes. In a 5-year study, we are using
a randomized experimental design to assign children to treatment in either a continuum
of care or “treatment as usual” model. The evaluation follows children and their families
over several years using standardized instruments to measure changes. We are tracking
emotional and behavioral functioning, stability of living environment, caregiver strain,
family strengths, and satisfaction with services. The evaluation also looks at how the
continuum of care developed during the 5-year period. Finally, Macro is doing a
cost-benefit analysis to assess cost savings or benefits associated with the continuum of
care model.
 Program
Evaluation Center
For the Minority Substance Abuse and Human Immunodeficiency Virus
(HIV) Prevention Initiatives, Macro serves as the Program Evaluation
Center for interventions to help prevent substance abuse and high-risk
sexual behaviors among minority populations. Working with an evaluation
partner, we are using a quasi-experimental design to measure outcomes.
The design provides for examination of the overall effectiveness
of 15 adult intervention programs and 6 youth intervention programs.
It also allows us to analyze effects for different groups of program
participants. Common measures used across the study sites provide
individual outcome data on risk and protective factors, substance
use, and sexual behaviors. Service data are collected on individual
participants in the intervention initiatives to measure program
dosage. Data collection is in progress. Findings from this 3-year
study will be reported in 2005.
|