Regression Discontinuity Design

Cutoffs, Local Average Treatment Effects, and Sharp vs. Fuzzy Designs

Professor Benjamin Noble

Politics Pays?

Higher Ed Dive headline: Ben Sasse hit the jackpot with his University of Florida contract. — Ben Sasse, via Higher Ed Dive

Photo of Nigel Lawson, former British Chancellor of the Exchequer. — Nigel Lawson

Eggers and Hainmueller (2009)

Eggers

Hainmueller

The Problem

“MPs are not randomly selected from the population (which is unfortunate for researchers, but arguably beneficial for citizens), so a comparison of MPs’ income or wealth with that of a peer group outside politics is likely to reflect factors that led MPs to gain political office as well as the value of political office itself.” (Eggers and Hainmueller 2009, 513–14)

The Solution

Compare politicians who narrowly won to politicians who narrowly lost.
A landslide winner and a big loser are very different.
Near 50% (e.g., 50.1 vs 49.9), winning is essentially luck (e.g., rain, traffic).
Near winners and near losers are almost randomly assigned.

The Evidence

Eggers and Hainmueller Figure 4: scatterplot of wealth at death versus vote share margin for Conservative candidates, with a visible jump at the zero margin cutoff. — Eggers and Hainmueller (2009)

Am I a Good Teacher? (Redux)

Good: score on a 100-point data analysis test.
Suppose we have a binary treatment variable:

D_i = \begin{cases} 1 & \text{if } i \text{ takes POLI 170} \\ 0 & \text{if } i \text{ does not take POLI 170} \end{cases}

Suppose we have an outcome variable, Y_i, person i’s score on the test.

Test Scores by POLI 30 Grade

Suppose: to take this class, you must have a POLI 30 score ≥ 75.
Everyone who applies (admitted or not) takes the same final exam afterward.
A problem: We are creating selection bias. Students in treatment (taking the class) are better at statistics.
A naive comparison of a POLI 30 student who scored 50 vs. 90 is not credible.

Scatterplot of POLI 170 final score versus POLI 30 score. Circles are students who did not take POLI 170; triangles took POLI 170. A dashed vertical line marks the cutoff at 75.

Looking Near the Cutoff

Students who scored 74 vs. 76 are essentially the same. Tiny chance differences determine which side of the line they land on.
Regression Discontinuity Design (RDD): exploit the cutoff by looking at near-cutoff observations that are quasi-randomly assigned.

Same scatterplot with students near the cutoff highlighted: yellow circles for students just below the cutoff who did not take the class, and blue triangles for students just above the cutoff who took it. All other students are gray.

Some Notation and Terminology

Forcing variable / running variable (e.g., POLI 30 grade): c_i
Treatment assignment (taking POLI 170):

D_i = \begin{cases} 1 & \text{if } c_i \geq c^* \\ 0 & \text{if } c_i < c^* \end{cases}

No overlap: every treated unit has c_i \geq c^*; every control unit has c_i < c^*. No value of c_i ever appears in both groups.

The Naive Regression

Scatterplot with gray circles for students who did not take the class and gray triangles for students who did. The lower dashed yellow fitted line represents the control group; the upper solid blue fitted line represents the treatment group.

lm(post_score ~ took_class + pre_score, data = df)

Term	Estimate	Std. Error	p-value
(Intercept)	-9.34	2.68	< .001
took_class	13.96	1.05	< .001
pre_score	0.92	0.04	< .001

LATE: Local Average Treatment Effect

The “true” treatment effect: LATE = \mathbb{E}[Y_i^1 - Y_i^0 \mid c_i = c^*]
We have no overlap at c^*, so we estimate an outcome model for values close to the cutoff: Y_i = \beta_0 + \beta_1 D_i + \beta_2 c_i + \varepsilon_i
The estimated LATE is \widehat{LATE} = \hat{\beta}_1.

Restricting to the Window

Scatterplot with far observations grayed out and near-cutoff observations highlighted as yellow circles for controls and blue triangles for treated students. A dashed yellow control line and solid blue treatment line cover the 72 to 77 window.

lm(post_score ~ took_class + pre_score,
  data = df %>% filter(between(pre_score, 72, 77)))

Term	Estimate	Std. Error	p-value
(Intercept)	20.24	59.22	0.733
took_class	18.64	2.31	< .001
pre_score	0.54	0.81	0.504

Assumptions and Limitations

We need a lot of data around the discontinuity.
Local randomization near the cutoff: (Y_i^1, Y_i^0) \perp D_i for c_i \in (c^* - a, c^* + a).
No sorting / continuity at the cutoff (violations: other treatments, manipulation of scores).

McCrary Density Test (2008)

If people sort around the cutoff, we might see it in the density of the running variable.
Limitation: only catches obvious violations. Complement it with covariate checks and placebo tests at other cutoffs.

Two density plots of the running variable (income) around a cutoff at 14: panel C shows a smooth continuous density, panel D shows a sharp drop just above the cutoff indicating sorting.

Bandwidths…

How close is “close”?
One solution: show many “bandwidths” and hope they generally give you the same results.

Lower	Upper	LATE est.
74	76	17.76
73	77	19.34
72	78	18.81
71	79	18.37
70	80	18.34
69	81	18.53
68	82	18.98
67	83	19.64
66	84	20.63
65	85	20.06

Sharp vs Fuzzy RDD

Sharp: cutoff perfectly separates treatment and control.
Fuzzy: cutoff generally, but imperfectly, separates treatment and control.
- Think of this like an instrumental variables / encouragement design.

Hoekstra (2009) plot: enrollment rate against SAT points above or below admission cutoff. Probability of enrollment jumps from near zero to about 0.5 at the cutoff but is not 0/1. — Hoekstra (2009) via Cunningham (2021)