ICF Macro's evaluation of systems of care for the Center for Mental Health Services has resulted in an extensive series of data sets that require the use of complex and combined multivariate analytic techniques. The data gathered for the child and family outcome study are collected longitudinally and analyzed employing generalized linear models, fixed effects and random effects regression models. Many indicators collected through evaluation are categorical and require the utilization of such techniques as logistic regression, multinomial logistic regression, and ordered Probit regressions. Person- and item-level data reduction techniques, including latent class analysis, factor analysis, and cluster analysis, are often utilized to better understand the interrelations within and across construct indicators. We collect both process and outcome measures at the system and client levels, and use this information in multilevel analyses that model the degree to which community- and system-level change influences child and family outcomes (i.e., hierarchical linear modeling). Survival analysis methods are employed to look at various administrative data and agency records (i.e., service records, criminal records) that are episodic in nature. We use log-transformed and gamma regression models to analyze cost data that are often skewed. The nature and structure of the various national evaluation data sets has required a sophisticated and state-of-the-art approach to creative and cutting edge data analytic techniques.
Evaluation of Substance Abuse Prevention Initiatives for High-Risk Youth
ICF Macro employed a multi-stage analysis plan to evaluate a demonstration
program aimed at preventing alcohol, tobacco, and illicit drug use
among at-risk youth. Using a quasi-experimental design, we compared
some 6,000 program participants with 4,000 comparison youths in
48 demonstration programs. The analysis included reliability and
validity of the instrument scales, cross-site measurement stability,
attrition analysis, correlational analysis of baseline variables,
and treatment and control group equivalence analysis. We analyzed
each data set—participant surveys, individual exposure to
program activities, and program characteristics—and then the
integrated data set. Analysis of the integrated data set involved
multiple regression, LISREL, structural equation modeling, and hierarchical
linear modeling.
Customer Satisfaction Survey
ICF Macro performs quarterly statistical analyses of a customer
satisfaction survey for the Defense Logistics Agency (DLA). We provide
overall results along with results for each of 10 service units.
Our analysis includes basic descriptive statistics, cross-tabulation,
regression and correlation analyses, and cluster and factor analyses
as indicated by our client’s needs. In addition, we provide
the DLA with our proprietary Customer Loyalty Plus® approach,
which allows the client to efficiently set priorities and allocate
resources to improve customer satisfaction. To complement the statistical
analysis, we code respondents’ verbatim comments using Macrosaic©,
which analyzes the comments by frequency, relates the data to the
quantitative data, and generates descriptive reports.
|