Research procedure
Escalate exam audits when discrepancies persist
- Regrade a random 10% exam sample
- If discrepancy exceeds 5%, audit another 10%
- If audited average remains above 5%, regrade 100%
Illustrative summary of the source-described tiered exam audit. Central auditors compare their grades with teachers’ original grades; escalation applies to that teacher’s exams.
See source →How
When to use
When local staff grade exams that determine their own performance raises, organizations face high risks of grade manipulation and score inflation. Centralized regrading of every test paper is often too expensive and time-consuming for large school networks. A tiered sampling audit provides reliable quality assurance by escalating inspection intensity only when initial grade discrepancies exceed tolerance thresholds.
Audit locally graded standardized exams using a multi-stage random sampling protocol with automatic escalation triggers.
- Select a random 10% sample of all student exams graded by local teachers across participating schools at the end of the term.
- Regrade the sampled exam papers independently using central research team auditors.
- Calculate the percentage score discrepancy between the original teacher grade and the auditor grade for each audited paper.
- Trigger a secondary audit of an additional 10% of that teacher's exams if the initial score discrepancy exceeds 5%.
- Regrade 100% of the teacher's exams if the average score discrepancy across audited papers remains above 5%.
Why it works
Escalating to full regrading when discrepancies persist catches intentional gaming without overburdening central staff with routine regrading.
Example
A private school system in Pakistan audited locally graded June standardized exams across core subjects in 234 schools. Auditors regraded a 10% sample of test papers, triggering secondary audits whenever grades deviated by more than 5%, which kept grade manipulation small and mostly limited to bumping failing scores up to passing marks.
About the study
Participants
10% sample of student standardized exams across core subjects (English, Urdu, math, science, economics) in 234 private schools in Pakistan (grades 4–13).
Study Design
Randomized controlled trial (RCT) testing implementation fidelity through a tiered audit algorithm (initial 10% sample, secondary 10% sample if discrepancy > 5%, full 100% regrade if average discrepancy > 5%) comparing flat, subjective, and objective incentive contracts.
Results
Overall grade manipulation remained small and was primarily driven by teachers bumping student scores from just failing to just passing, with zero statistically significant difference in grading accuracy across control, subjective, and objective treatment arms.