List of Figures and Tables
Preface
Acknowledgments
About the Editors
PART I. SETTING UP FOR EVALUATION
Chapter 1. Introduction
Background: Challenges and Opportunities
Evaluation and Health Care Delivery System Transformation
The Global Context for Considering Evaluation Methods and Evidence-Based Decision Making
Chapter 2. Setting the Stage
Typology for Program Evaluation
Planning an Evaluation: How Are the Changes Expected to Occur?
Developing Evaluations: Some Preliminary Methodological Thoughts
Prospectively Planned and Integrated Program Evaluation
Chapter 3. Measurement and Data
Selecting Appropriate Measures
PART II. EVALUATION METHODS
Chapter 4. Causality and Real-World Evaluation
Evaluating Program/Policy Effectiveness: The Basics of Inferring Causality
Three Key Treatment Effects
Statistical and Real-World Considerations for Estimating Treatment Effects
Chapter 5. Randomized Designs
Randomized Controlled Trials
Randomized Designs for Health Care
Chapter 6. Quasi-experimental Methods: Propensity Score Techniques
Dealing With Selection Bias
Comparison Group Formation and Propensity Scores
Regression and Regression on the Propensity Score to Estimate Treatment Effects
Chapter 7. Quasi-experimental Methods: Regression Modeling and Analysis
Interrupted Time Series Designs
Comparative Interrupted Time Series
Difference-in-Difference Designs
Instrument Variables to Estimate Treatment Effects
Regression Discontinuity to Estimate Treatment Effects
Fuzzy Regression Discontinuity Design
Additional Considerations: Dealing With Nonindependent Data
Chapter 8. Treatment Effect Variations Among the Treatment Group
Context: Factors Internal to the Organization
Evaluation Approaches and Data Sources to Incorporate Contextual Factors
Context: External Factors That Affect the Delivery or Potential Effectiveness of the Treatment
Individual-Level Factors That May Cause Treatment Effect to Vary
Methods for Examining the Individual Level Heterogeneity of Treatment Effects
Importance of Incorporating Contextual Factors Into an Evaluation
Chapter 9. The Impact of Organizational Context on Heterogeneity of Outcomes: Lessons for Implementation Science
Context for the Evaluation: Some Examples From Centers for Medicare and Medicaid Innovation
Evaluation for Complex Systems Change
Frameworks for Implementation Research
Organizational Assessment Tools
Analyzing Implementation Characteristics
PART III. MAKING EVALUATION MORE RELEVANT TO POLICY
Chapter 10. Evaluation Model Case Study: The Learning System at the Center for Medicare and Medicaid Innovation
Step 1: Establish Clear Aims
Step 2: Develop an Explicit Theory of Change
Step 3: Create the Context Necessary for a Test of the Model
Step 4: Develop the Change Strategy
Step 6: Measure Progress Toward Aim
Chapter 11. Program Monitoring: Aligning Decision Making With Evaluation
Cases: Examples of Decisions
Evidence Thresholds for Decision Making in Rapid-Cycle Evaluation
Chapter 12. Alternative Ways of Analyzing Data in Rapid-Cycle Evaluation
Statistical Process Control Methods
Regression Analysis for Rapid-Cycle Evaluation
A Bayesian Approach to Program Evaluation
Chapter 13. Synthesizing Evaluation Findings
Meta-evaluation Development for Health Care Demonstrations
Chapter 14. Decision Making Using Evaluation Results
Research, Evaluation, and Policymaking
Program/Policy Decision Making Using Evidence: A Conceptual Model
Multiple Alternatives for Decisions
A Research Evidence/Policy Analysis Example: Socioeconomic Status and the Hospital Readmission Reduction Program
Other Policy Factors Considered
Advice for Researchers and Evaluators
Chapter 15. Communicating Research and Evaluation Results to Policymakers
Suggested Strategies for Addressing Communication Issues
Other Considerations for Tailoring and Presenting Results
Closing Thoughts on Communicating Research Results
Appendix A: The Primer Measure Set
Appendix B: Quasi-experimental Methods That Correct for Selection Bias: Further Comments and Mathematical Derivations
An Alternative to Propensity Score Methods
Assessing Unconfoundedness
Using Propensity Scores to Estimate Treatment Effects
Unconfounded Design When Assignment Is at the Group Level
Index