Source · aikyam school

Targeting Labour Market Programmes – Results from a Randomized Experiment

Original source

Stefanie Behncke; Markus Frölich; Michael Lechner

Targeting Labour Market Programmes – Results from a Randomized Experiment (2009)

Published research article · 145(3), 221–268

Search for this source ↗

Permission for School’s use is unestablished.

Full source notice

© Swiss Society of Economics and Statistics 2009, Vol. 145 (3) 221–268

Used in these cards

Design algorithmic implementation protocols that combine caseworker discretion with explicit compliance incentives or mandatory justification requirements when deviating from statistical predictions.

"Without strong incentives or coercion, caseworkers may ignore or sabotage a system that is intended to provide information only."

Structure field experiments of organizational decision-support tools by randomizing frontline staff rather than end clients to eliminate informational spillovers and cross-treatment extrapolation.

"A randomization at the level of the caseworker was preferred to a randomization at the level of the employment office or at the level of the unemployed person... A randomization at the level of the unemployed person, on the other hand, would have led to the problem that a caseworker would receive employment predictions for some of his clients but not for others... produce spill over effects."

Re-calculate individual potential outcome predictions biweekly using updated elapsed duration and labor market indicators to adjust treatment recommendations dynamically.

"The predictions were updated every second week by incorporating new information on time varying covariates (in particular unemployment duration). This is a big advantage vis-à-vis simple profiling models as it takes into account that the optimal time when a labour market programme should start may also vary across individuals."

Aggregate highly disaggregated intervention types into broader functional categories that guarantee valid choice sets for all client demographics while preserving statistical estimation power.

"By defining a category language skills training which includes German, French and foreign language courses, this category becomes feasible for every jobseeker, and the X_it characteristics (e.g. mother tongue, profession) define which type of language course or further training is appropriate."

Address frontline job security fears, involve staff unions during software design, and establish binding privacy firewalls before introducing decision-support tools in public sector agencies.

"SOMS, however, was never implemented mainly because of data security concerns and because of resistance from frontline caseworkers who perceived SOMS as a threat to their own job security... The SOMS database was deleted in 2002, before SOMS ever was tried in the field."

Instrument actual software usage with randomized assignment status to estimate local average treatment effects when software adoption is voluntary and non-compliance is one-sided.

"Since it was impossible for the members of the control group to download the predictions the monotonicity condition of IMBENS and ANGRIST (1994) is satisfied by definition, and this also means that the local average treatment effect (LATE) is the same as the average treatment effect on the treated (ATET) because the treated are the compliers."

Define experimental treatment status using initial caseworker assignment while running sensitivity robustness checks on non-switcher cohorts to verify that administrative rotation rules do not bias impact estimates.

"In many offices, the caseworker is changed if the jobseeker stays unemployed for more than 6 or 8 months, with the intention to introduce new ideas in the job search process... In most of the analyses, we will define treatment status of the jobseeker as time-invariant according to the randomization status of the first caseworker."

Link national unemployment benefit administration registers with social security pension records to build longitudinal monthly panel datasets for non-parametric counterfactual prediction.

"This data set was matched with the complete monthly information from the social security and pensions system (AHV) for the period January 1990 to December 2002. These combined data sources contain very detailed information on registration and de-registration of unemployment, benefit payments and sanctions, participation in ALMP, eleven years employment histories with monthly information on earnings and employment status."

Partition multidimensional algorithmic output into statistical confidence tiers using Multiple Comparison with the Best routines before displaying recommendations to human decision-makers.

"The statistical precision of the estimates was also conveyed to the caseworker in that the set of all programmes was divided into three groups: The significantly best programmes, the intermediate programmes, and the worst programmes... estimated by Multiple Comparison with the Best (MCB) routines."

Survey frontline operational staff to identify misalignment between personal caseworker goals and statutory legislation, then restructure organizational incentives to align agent decisions with principal objectives.

"Indeed, when asked in a written questionnaire, around 40% of Swiss caseworkers reported 'improving their clients personal situation', while 35% reported 'long-run reintegration' and 25% 'rapid reintegration' as their primary goal for low qualified unemployed (FRÖLICH et al., 2007)."

Transition public service allocation tools from passive risk profiling models to multi-treatment targeting models that evaluate conditional potential outcomes across distinct interventions.

"Targeting is preferable to profiling if a variety of heterogeneous labour market programmes are offered... and if the long-term unemployment risk is not highly correlated with programme impacts."

Calculate individual potential employment outcomes using disaggregated administrative data, and integrate statistical prediction sets into caseworker workflow interfaces alongside explicit compliance monitoring protocols.

"The evaluation results show that caseworkers did not change their behaviour in any significant way due to having access to the additional information. Caseworkers either decided to ignore the system or were over-confident in that their own experience clearly dominates any information that a statistical system might provide."

Disaggregate field evaluation samples into distinct stock and flow cohorts to isolate duration dependence and evaluate early-intervention effects independently from long-term unemployment dynamics.

"We therefore distinguish in the later analyses between a stock and a flow sample. The stock sample contains all jobseekers who were already registered at the beginning of the field study... The flow sample contains all jobseekers who entered later."