Attrition Independence from Assigned Treatment Effectiveness

In Simple Words

In a drug study, if patients assigned to less effective doctors quit the trial far more frequently than those assigned to top doctors, comparing the patients who stayed creates a skewed picture. Verifying that missing data rates are equal regardless of doctor rating confirms the experiment remains fair.

Research question

In educational randomized controlled trials, if student drop-out rates or missing test score data correlate systematically with the effectiveness of assigned instructors, the remaining sample introduces attrition bias into causal effect estimates.

Participants

N = 31,439 randomized students in grades 4–8 and N = 8,657 randomized high school students across 6 urban school districts.

Study Design

Linear regression of outcome data availability indicator variables on assigned teacher effectiveness with randomization block fixed effects.

Results

Assigned teacher effectiveness had no statistically significant impact on student state test data availability (-0.012, SE = 0.033), 4th–8th grade MET project outcome availability (-0.070, SE = 0.143), or high school MET project outcome availability (-0.077, SE = 0.144), establishing that sample attrition was independent of treatment assignment.

Takeaway

Regress outcome data availability indicators on assigned treatment ratings to test for attrition bias in multi-site randomized field trials.

Read alongside the research

Plain-English glossary

Original explanations, informed by the references below. Examples are illustrations, not data from the studies above. The research text stays unchanged.

Statistically significant

A label used when a result passes a chosen statistical threshold, often p < 0.05. It does not show how large or useful an effect is. A result that is not statistically significant does not prove there is no effect.

ExampleA study reports p = 0.03 against a 0.05 threshold. It passes that test, but the effect could still be too small to matter.

American Statistical Association · Statement on p-values ↗Read more about statistically significant →

Related findings

• Post-Randomization Peer Compositional Stability →Education

  If participants in a fitness study invite outside friends to join their workouts, researchers must verify that the incoming friends do not give certain workout groups an artificial boost compared to others.

• Academic Grade-Incentivized Curriculum Compliance →Education

  Linking software progress to a real school report card acts like a mandatory gate on a turnstile. Students must complete digital quizzes and modules because their software performance directly impacts their overall grade in an established class like computer science.

• Academic Remediation Inefficacy in Postsecondary Persistence →Education

  Standard developmental remediation is like forcing aspiring drivers to complete months of classroom parking simulations before letting them drive on real roads. Because the simulations take time and money without moving drivers closer to their destination, most drivers become discouraged and quit driving before ever reaching the main highway.

• Administrative Algorithmic Dropout Targeting →Education

  Instead of waiting for students to start failing classes, administrators analyze prior middle school report cards and standardized admission test scores before the academic year begins to calculate an individual risk score. This functions like a health screening questionnaire at a hospital lobby that flags patients with elevated blood pressure before an acute cardiac event takes place.