When always-submitters have stronger baseline ability than conditional submitters, selective entry into submission can only pull down average treated scores.

Download image · 1200 × 630 PNG

Research steps

Bound quality effects under stated assumptions

  1. Collect submission decisions and proposal quality grades
  2. Apply no-defiers and better-always-submitters assumptions
  3. Compare submitter quality to derive a lower bound

Illustrative summary of the source-described procedure. The source assumes peer interaction never deters submission and always-submitters have no lower baseline quality than treatment-only submitters. The lower bound depends on both assumptions.

See source →

How

When to use

Evaluators need to estimate program impact on work quality when treatment changes participation rates. Simple comparisons between treated and control completers suffer from selection bias because lower-performing participants drop out or complete at different rates. Non-parametric selection-bias decomposition isolates the true treatment effect by accounting for voluntary participation changes.

Decompose observed outcome differences between treated and control completers using conditional selection bounds and participant behavioral assumptions.

  1. Collect submission decisions and quality grades from treated and control groups at the end of the intervention.
  2. Apply the no-defiers assumption that peer interaction does not cause any individual to withhold a proposal they would have submitted in control.
  3. Partition the population into always-submitters, never-submitters, and treatment compliers based on observed group submission rates.
  4. Apply the better-always-submitters assumption that always-submitters possess baseline quality no lower than individuals who submit only when treated.
  5. Compute the difference in average quality scores between treated and control submitters to establish a lower bound for the true unconditional quality effect.

Why it works

A positive conditional quality effect therefore guarantees a positive true impact across the full population.

Example

Researchers evaluated proposal quality among 1322 submitters out of 3333 entrepreneurs in five large African nations. Under the no-defiers and better-always-submitters bounds, virtual-within peer networking increased business proposal quality among motivated milestone completers by 0.138 points on a 1-to-5 scale.

About the study

Participants

3333 entrepreneurs in the Large-Country Sample (1322 submitted proposals evaluated by 15 African judges).

Study Design

Non-parametric selection-bias decomposition assuming No Defiers (ND) and Better Always Submitters (BAS) to evaluate conditional quality scores.

Results

Virtual-within peer networking produced a statistically significant positive effect on business proposal quality among motivated Milestone 0 completers (\beta = 0.138, p < 0.05 in Large Countries; \beta = 0.427, p < 0.05 in Uganda).

Source

See source →