Randomizing puzzle set versions between sessions prevents referring workers from giving their candidates answer keys or solution shortcuts.

Download image · 1200 × 630 PNG

Research procedure

Randomise puzzle versions by session

  1. Prepare similar puzzle sets A and B
  2. Randomly assign a set to each laboratory session
  3. Test participants; record set version for analysis

Illustrative summary of the source-described session-level puzzle assignment. Set version is included as a control in second-stage analysis; no performance result is shown.

See source →

How

When to use

Randomize puzzle problem sets across consecutive testing laboratory sessions when returning participants might share solution details with new candidates. This eliminates information leakage and cheating when evaluating referred candidates on cognitive tasks. Use this protocol when testing candidate skill across peer social networks.

Randomize puzzle set versions across laboratory sessions to prevent referring workers from communicating puzzle answers to referred candidates.

  1. Develop two parallel sets of geometric quilt puzzles featuring similar rules and difficulty levels.
  2. Designate one puzzle version as Set A and the second version as Set B.
  3. Randomize whether Set A or Set B is assigned to each laboratory session when original participants arrive.
  4. Administer the assigned puzzle set version during the candidate's testing session.
  5. Record the specific puzzle set version assigned to each participant as a control variable in second-stage econometric specifications.

Why it works

Including the puzzle version as a control accounts for any minor residual variance in task difficulty.

Example

Field staff randomized two puzzle set versions across testing laboratory sessions in Kolkata, India, involving 561 original participants and 407 referrals. Controlling for puzzle type confirmed that cognitive task scores reflected candidate ability rather than solution sharing.

About the study

Participants

561 original participants and 407 referred male candidates in Kolkata, India.

Study Design

Session-level randomization of two equivalent puzzle sets combined with regression control for puzzle set version.

Results

Maintained experimental integrity across sessions, showing a non-significant baseline correlation between assigned puzzle type and treatment status (p = 0.91) while controlling for puzzle set version in all specifications.

Source

See source →