Research steps
Classify counsellors using past client dropout
- Extract 2010 records; group by counsellor
- Calculate first-meeting dropout proportions
- Compare rates with each centre’s average
Illustrative summary of the source-described historical classification before the trial. Below-centre-average attrition identifies the higher-quality category. These are prior dropout measures, not a new causal estimate of counsellor quality.
See source →How
When to use
Public employment agencies assign counselors to jobseekers without knowing which staff members effectively engage clients and which cause high dropout rates. Measuring individual counselor performance is difficult when participant behavior is confounded by ongoing program incentives. This method evaluates caseworker quality by analyzing historical client dropout rates from administrative records prior to new policy rollouts.
Measure individual counselor quality using past administrative records of client program attrition following initial meetings.
- Program evaluators extract administrative jobseeker records from national employment agency databases for the year preceding a new program launch.
- Evaluators group past jobseekers by their assigned individual counselor within each local youth employment center.
- Evaluators calculate the proportion of jobseekers who permanently dropped out of guidance after attending their first meeting with each counselor.
- Staff de-mean individual counselor dropout rates at the local center level to control for regional differences in client demographics.
- Evaluators classify counselors as high quality if their historical client dropout proportion falls below their center's average attrition rate.
Why it works
Historical first-meeting attrition provides an unconfounded indicator of counselor effectiveness prior to introducing new financial incentives.
Example
In France, evaluators matched 3,710 trial participants to their assigned counselors using 2010 administrative records, classifying counselors with below-average client dropout rates after the first meeting as high quality; high-quality counselors generated significantly more first-quarter training proposals (88.3 vs 77.7) and higher baseline human capital investment index scores (9.50 vs -7.33).
About the study
Participants
Subgroup match of 3,710 young jobseekers aged 18–22 across 82 French Job Youth Centers with caseworker administrative records from 2010.
Study Design
Randomized controlled trial analyzing treatment effect heterogeneity based on historical caseworker client dropout rates in 2010.
Results
Assigning jobseekers to high-quality caseworkers resulted in significantly higher first-quarter training proposals (88.3 vs 77.68, p < 0.01) and higher baseline human capital investment index scores (9.50 vs -7.33), though treatment effects on training completions did not differ significantly between high- and low-quality caseworkers (-1.36 vs +4.85, p > 0.10).