Stock vs. Flow Sampling in Program Evaluation

In Simple Words

Imagine testing a new study method in a university. The "stock sample" consists of upperclassmen who have already developed specific study habits and passed or failed several courses. The "flow sample" consists of incoming freshmen starting fresh on day one, allowing researchers to see how the method works from the very beginning.

Research question

Evaluating policy interventions on existing registered clients (stock sample) introduces duration bias because these individuals have already accumulated varying lengths of unemployment and established caseworker habits. Evaluating newly registered clients (flow sample) provides cleaner estimates of initial intervention timing but risks non-random caseworker assignment during field rollout.

Participants

N = 22,758 jobseekers in the stock sample (registered before pilot inception) and N = 12,632 jobseekers in the flow sample (registered during pilot rollout) across Swiss employment offices.

Study Design

Comparative RCT subgroup analysis evaluating treatment effects and compliance rates separately for stock versus flow jobseekers to control for duration dependence and administrative assignment changes.

Results

Short-term compliance rates remained identical across both subgroups (11% stock vs 14% flow for Definition 1; 29% stock vs 30% flow for Definition 2), confirming that timing of prediction availability did not alter caseworker compliance behavior.

Takeaway

Disaggregate field evaluation samples into distinct stock and flow cohorts to isolate duration dependence and evaluate early-intervention effects independently from long-term unemployment dynamics.

Read alongside the research

Plain-English glossary

Original explanations, informed by the references below. Examples are illustrations, not data from the studies above. The research text stays unchanged.

Randomized controlled trial

A study that uses chance, like drawing lots, to decide who receives each option being compared. Researchers compare the groups’ outcomes to estimate the effect of those options.

ExampleA lottery assigns volunteers to a training programme or a comparison group. Researchers then compare their outcomes.

GET-IT glossary · Randomized study ↗Read more about randomized controlled trial →

Related findings

• Active Tracking Attrition Mitigation →Methods & evidence

  Dispatching field teams directly to children's homes when they miss school testing operates like a home delivery service tracking down uncollected mail to ensure every package is accounted for.

• Commercial Ad Platform Algorithmic Randomization Imbalance →Methods & evidence

  Allowing a commercial ad algorithm to handle randomization is like letting a profit-driven tournament host pick matching opponents. The platform optimizes for click rates rather than making sure both starting teams are equally balanced.

• Multi-Source Administrative-Survey Data Triangulation →Methods & evidence

  Think of monitoring a patient's health by combining heart rate monitors and blood tests with direct interviews about how they feel. Combining automated medical readings with verbal feedback provides a complete, accurate picture of health rather than relying on either measurement alone.

• Adversarial Non-Stationarity Bounds in Multi-Armed Bandits →Methods & evidence

  Imagine playing a card game where an opponent can secretly alter the deck rules between rounds. If you assume the deck never changes, early luck might trick you into a permanently failing strategy.