Exogenous variation in survey contact effort and completion incentives shifts response probabilities without altering candidates' actual job search or employment choices.

Download image · 1200 × 630 PNG

Research steps

Adjust for survey non-response

  1. Randomise contact effort, alerts and incentives
  2. Record completion; model survey response
  3. Include correction in outcome regressions

Illustrative summary of the source-described procedure. Randomised features include call order, up to 20 attempts and a 100 PKR incentive. The two-stage correction relies on these features affecting response without changing underlying outcomes.

See source →

How

When to use

Field surveys often suffer from differential non-response between treatment and control groups, creating sample selection bias. Randomizing survey contact features generates instrumental variables that predict survey completion without altering underlying program outcomes. It is intended for researchers and evaluators measuring labor market behavior subject to survey attrition.

Correct for survey non-response selection bias by randomizing contact intensity, advance messaging, and financial incentives to construct instruments for a selection correction model.

  1. Assign target survey respondents randomly to varying levels of contact effort, including total call attempt caps and scheduling priority.
  2. Cross-randomize advance text message alerts and monetary incentives across call attempts.
  3. Record respondent completion across survey modules.
  4. Estimate a first-stage selection model regressing survey response status on the randomized contact and incentive features.
  5. Include the estimated selection correction term in second-stage regressions of primary outcomes on treatment assignment.

Why it works

This satisfies instrumental variable exclusion restrictions, allowing unbiased estimation of treatment effects despite differential survey attrition.

Example

In Lahore, Pakistan, researchers randomized call attempt caps (up to 20 attempts), advance text alerts, scheduling priority, and a 100 PKR completion incentive across follow-up surveys of 9,823 jobseekers. Extra call attempts increased survey response rates by 6.5 to 11.0 percentage points, enabling selection-corrected estimation of treatment effects on off-platform search and employment.

About the study

Participants

N = 9,823 jobseekers (survey modules completed by 2,081 to 4,643 respondents) in Lahore, Pakistan; raw survey response rates were 53.3% in control and 42.7% in treatment.

Study Design

Randomized survey administration experiment using extra call attempts, advance text alerts, call timing, and financial incentives as excluded instruments in a two-stage DiNardo et al. (2021) selection correction model.

Results

Extra call attempts increased survey module completion by 6.5 to 11.0 percentage points (p < 0.001, F-statistics 79.1 to 149.9), recovering selection-adjusted treatment effects on off-platform search (-0.01078, p = 0.603) and work (+0.01081, p = 0.589).

Source

See source →