Phased Treatment Enablement Protocol

In Simple Words

Requiring participants to complete a mandatory multi-week baseline trial period before enabling experimental features ensures that only active, compliant users enter the randomized treatment evaluation pool.

Research question

In digital field experiments, high initial user drop-off immediately following app download can pollute randomized treatment groups with inactive participants.

Participants

N = 8,248 enrolled participants filtered down to N = 4,458 retained migrant respondents across 30 follow-up weeks.

Study Design

4-week pre-treatment qualification phase requiring response to at least 2 out of 4 weekly surveys prior to randomization, followed by staggered treatment feature rollout in weeks 6, 7, 8, or 9.

Results

The phased enablement protocol effectively isolated non-complying recruits without introducing observable baseline trait imbalances (p > 0.10 across balance tests on gender, age, income, and remittance history).

Takeaway

Enforce a multi-week active user screening period prior to feature rollout in digital RCTs to prevent differential attrition from skewing experimental results.

Read alongside the research

Plain-English glossary

Original explanations, informed by the references below. Examples are illustrations, not data from the studies above. The research text stays unchanged.

Follow-up

Collecting more information about participants later in a study to see what has happened to them. The term can also mean the length of time researchers keep track of them.

ExampleResearchers interview the same workers three months after a programme ends. That later interview is a follow-up.

GET-IT glossary · Follow-up ↗Read more about follow-up →

Baseline

The starting measurements in a study. Researchers use these to compare what participants were like at the start with what happens later.

ExampleBefore a training programme starts, researchers measure workers’ earnings. That first measurement is the baseline.

GET-IT glossary · Baseline characteristics ↗Read more about baseline →

Related findings

• Active Tracking Attrition Mitigation →Methods & evidence

  Dispatching field teams directly to children's homes when they miss school testing operates like a home delivery service tracking down uncollected mail to ensure every package is accounted for.

• Commercial Ad Platform Algorithmic Randomization Imbalance →Methods & evidence

  Allowing a commercial ad algorithm to handle randomization is like letting a profit-driven tournament host pick matching opponents. The platform optimizes for click rates rather than making sure both starting teams are equally balanced.

• Multi-Source Administrative-Survey Data Triangulation →Methods & evidence

  Think of monitoring a patient's health by combining heart rate monitors and blood tests with direct interviews about how they feel. Combining automated medical readings with verbal feedback provides a complete, accurate picture of health rather than relying on either measurement alone.

• Adversarial Non-Stationarity Bounds in Multi-Armed Bandits →Methods & evidence

  Imagine playing a card game where an opponent can secretly alter the deck rules between rounds. If you assume the deck never changes, early luck might trick you into a permanently failing strategy.