Reversing the topic order creates an internal control group at midterm where both arms have received identical briefing materials and off-site exposure, differing only in which topic they discussed on-site.

Download image · 1200 × 630 PNG

Research design

Deliberate topics in opposite orders

Randomly assign small groups to topic-order streams

Airport-first: n = 64
Morning: airport. Midterm questionnaire. Afternoon: revenue sharing.
Revenue-sharing-first: n = 68
Morning: revenue sharing. Midterm questionnaire. Afternoon: airport.

Illustrative summary of the source’s split-half crossover. Both streams receive briefings and answer the same questions at baseline, midterm and completion. The text reports 133 attendees alongside the stream counts shown; it does not reconcile them. No results are shown.

See source →

How

When to use

Standard public consultation evaluations measure overall opinion changes without distinguishing formal on-site discussion from background reading or off-site conversations. Researchers use this method to isolate the specific impact of structured on-site deliberation from other event components. It provides policy analysts with rigorous causal evidence on how deliberation affects public opinion across issues with differing initial salience.

Randomly assign participants into two treatment streams that deliberate two distinct policy topics in opposite sequential orders, measuring outcomes after each stage.

  1. Draw a random sample of citizens and recruit them to attend a multi-topic deliberative event on two policy issues.
  2. Divide participants randomly into two equal treatment streams across small discussion groups at the event.
  3. Schedule the first treatment stream to deliberate the first issue during the morning session and the second issue during the afternoon session.
  4. Schedule the second treatment stream to deliberate the second issue during the morning session and the first issue during the afternoon session.
  5. Administer identical questionnaires at baseline, after the first deliberative session at midterm, and at event completion.
  6. Compare outcome shifts between treatment streams at midterm to isolate the causal effect of on-site deliberation on each specific issue.

Why it works

Comparing outcomes between streams at midterm isolates the specific effect of formal on-site deliberation from anticipatory reading and informal discussion.

Example

Researchers assigned 133 participants to 16 small groups across two streams (64 in the airport-first stream and 68 in the revenue-sharing-first stream) to deliberate airport expansion and regional revenue sharing in New Haven, Connecticut. At the midterm evaluation, participants in the revenue-sharing-first stream who deliberated revenue sharing showed a 0.313 mean rating shift toward voluntary tax sharing, compared to a -0.012 shift among airport-first participants who had only discussed airport expansion.

About the study

Participants

Initial interview sample of 1,032 residents from 15 towns surrounding New Haven, Connecticut, with 133 attending the event. Participants were 50.8% male, 49.2% female, 72.7% Caucasian, 12.1% African American, 3% Hispanic or Latino/a, and 90.2% registered voters, with an average age of 50.1 years.

Study Design

Split-half randomized crossover experiment with 16 small groups assigned to an airport-first stream (n = 64) or a revenue-sharing-first stream (n = 68) sequence, measuring policy attitudes, proximity to single-peakedness, and knowledge at baseline, midterm, and event completion.

Results

On revenue sharing, attitude shift toward voluntary tax sharing at midterm was 0.313 for the revenue-sharing-first stream versus -0.012 for the airport-first stream. Proximity to single-peakedness for revenue sharing at midterm reached 0.78 for the revenue-sharing-first stream versus 0.63 for the airport-first stream, and revenue sharing knowledge at midterm increased by 40.4% in the revenue-sharing-first stream versus 6.3% in the airport-first stream.

Source

See source →