RTI Center for Advanced Methods Development

Innovative evaluation methods to strengthen policies and programs

From health to education to social policy, when stakeholders invest in programs and innovations, they want to know if their investments produce the intended effects. For example:

• Did meditation during detention improve the school outcomes for kids who are often in detention?
• Did a state health care system save money and maintain or improve health outcomes when it changed its financing model?
• Under what circumstances do weight loss interventions work best within the Veterans Health Administration?

The RTI Center for Advanced Methods Development works to catalyze advances in research and move policy forward by focusing on innovative methods that produce evidence that is efficient, interactive, and actionable. By capitalizing on advances in data analysis and visualization, we can address complex challenges and deliver the answers our clients need.

Our experts draw from decades of experience analyzing and synthesizing scientific data as co-leaders of the RTI International-University of North Carolina Evidence-based Practice Center (RTI-UNC EPC). Under this effort, we have developed sound approaches to review bodies of complex scientific evidence and produce clear findings to inform health care practice and policy.

Grounded in this work, we are able to develop innovative approaches to program evaluation to answer critical questions about education outcomes, identify health care innovations that work, and advance the state of science.

Measuring Student Growth

Recognizing the challenges facing educators to measure student growth, RTI collaborated with researchers from the University of Michigan and the Bill & Melinda Gates Foundation to create a tool called the Evaluation Engine. The tool makes impact evaluation easier, less costly, and more rigorous.

Evaluating the Health Care Innovation Awards

We used a combination of rigorous approaches to analyze the Health Care Innovation Awards—a $1 billion initiative of the Centers for Medicare and Medicaid Innovation.

To provide policymakers with real-time information on what works, where, and why, we use meta-analysis, data visualization, Bayesian analysis, and qualitative analysis. Our interactive dashboard allows CMS policy makers to curate findings based on the data most relevant to them and their questions.

Bayesian Analysis to Understand Policy Impacts

Using Bayesian methods we provide rigorous analyses and an interactive tool that helps define the potential impacts of policy. Policymakers can see evaluation results and understand the probability of different levels of savings or other relevant policy impacts.

See how we’re applying Bayesian methods in support of policy evaluations.

Rolling Entry Matching

Program evaluations are often challenged by rolling entry of participants in treatment or other programs. As the composition of the treatment group changes, the evaluation design must also adjust to construct meaningful comparison groups in nonexperimental evaluations.

We developed a statistical method dubbed Rolling Entry Matching that provides reliable evidence even with a small number of participants who enroll over time.

With the analytic power of this approach, even in the absence of a controlled trial with a large cohort we can conduct meaningful evaluations that support client goals of finding and scaling innovations that are proven to improve patient health.

Qualitative Comparative Analysis (QCA) in Health Services Research

QCA methods use Boolean logic to allow researchers and policymakers to understand necessary and sufficient conditions for the existence of specific outcomes. For example, when the context for implementing an intervention varies by site, characterizing the site and implementation process can be helpful for improving processes.

Learn how we applied QCA to analyze the Move Program on behalf of the Veterans Health Administration.

Delivering Evidence-Based Answers to Critical Questions

By building on the experience and expertise of our team and affiliates, the RTI Center for Advanced Methods Development is working to catalyze advances in research ― focusing on innovative methods that strengthen evaluations and produce evidence that is efficient, interactive, and actionable.

Our clients and partners can be confident that evaluation results derived from our creative, rigorous, and cutting-edge methods are a sound basis for moving policy forward.