ECON 4309 · Quantitative Methods for Social Sciences · Reading companion

Instrumental Variables, in about 15 minutes

A plain-language guide to Chapter 19 of The Effect by Nick Huntington-Klein. Read this first, play with the two simulators, and then use the reading guide at the end to decide which parts of the chapter to read closely.

Pairs with Lectures 11–12: IV I & IIBook: Ch. 19~15 min

Note: This is the printable version. The interactive version, with the simulators, is on the course website.

1. The big idea in 60 seconds

The chapter opens with a statue. You can make one by taking a block of marble and chipping away everything that isn't the statue. Or you can pour metal into a mold that only lets the shape you want come through. Controlling for variables is the marble approach: you try to remove every back door one by one. Instrumental variables (IV) is the mold: you find one clean source of variation and use only that.

The ideal clean source is a randomized experiment, where a coin flip decides who gets treated, so the coin flip has no back doors. IV tries to mimic an experiment without running one. You look for something in the real world that pushes people into treatment more or less at random. That "something" is the instrument.

In Lecture 11's example, we want the effect of education on wages, but ability affects both and we can't measure it. The instrument is father's education. In the chapter's words, IV works in steps:

Use the instrument to explain the treatment and keep only that explained part (the part of your education that is predicted by your father's education).
Use the instrument to explain the outcome and keep only that explained part.
Compare the two explained parts. How much did wages move for each year of education that the instrument moved?

IV estimate = effect of the instrument on the outcome ÷ effect of the instrument on the treatment

Notice that this is the opposite of controlling. When you control for a variable, you throw away the part it explains. With an instrument, you keep only the part it explains, because that's the part with no back doors.

2. Eight words you need

We'll use Lecture 11's example throughout: education → wages, with unmeasured ability as the confounder and father's education as the instrument.

Instrument (Z)A variable that pushes the treatment around but has no other connection to the outcome.Father's education nudges how much school you get.

RelevanceThe instrument must actually move the treatment. This is the one assumption you can check with data.Do kids of more-educated fathers get more education? (Yes, strongly.)

Exclusion restrictionThe book calls it validity. Every path from the instrument to the outcome must go through the treatment. Lecture 11 splits this into excludability (no direct effect) and exogeneity (no shared causes).Does your father's education raise your wages in any way other than through your education?

First stageThe regression of the treatment on the instrument (plus controls).educ ~ fathereduc: how many extra years of school per year of father's schooling.

Reduced formThe regression of the outcome directly on the instrument. In an experiment, this is the "intent-to-treat" effect.wage ~ fathereduc

2SLSTwo-stage least squares. Stage 1 predicts the treatment from the instrument. Stage 2 regresses the outcome on that prediction. It's the most common way to do IV.Stage 2: wage ~ educ_hat

Compliers & LATECompliers are the people whose treatment actually responds to the instrument. IV gives a local average treatment effect (LATE) that is mostly about them.Lecture 12's bed nets: people who use a net only if the NGO gives them one.

Weak instrumentAn instrument that moves the treatment only a little. Dividing by a tiny first stage makes estimates swing wildly. Check the first-stage F-statistic.Lecture 11: social security number would be (at best) a hopelessly weak instrument for education.

3. Simulator: OLS vs. IV, strong vs. weak instruments

The data below are simulated, so we know the truth: each year of education raises the hourly wage by exactly $2.00. Hidden ability raises both education and wages, so plain OLS is biased upward. Father's education is the instrument. Its effect on education is the "strength" slider. You can also break the exclusion restriction by letting father's education affect wages directly.

Simulated workers: years of education vs. hourly wage (500 people)

Large dots are group averages after sorting people into five equal-sized groups by father's education. The IV line is driven by how those group averages move. OLS uses every dot, including the variation that comes from ability.

OLS lineIV (2SLS) line and the five group averagesTrue slope ($2.00)

Instrument strength: 0.50 extra years of school per year of father's schooling

Direct effect of father's education on wages: $0.00 per year

OLS estimate

—

IV estimate

—

True effect

2.00

First-stage F

—

Try 1: Leave the settings alone and click "Repeat 200 times." The OLS dots bunch tightly around the wrong number (ability bias). The IV dots are more spread out but centered on $2. IV trades precision for being right.
Try 2: Slide the strength down to about 0.06. The F-statistic falls below 10, the IV line swings around from sample to sample, and the repeated IV estimates scatter all over. Push it down to 0.02 and the middle of the IV estimates also drifts toward the OLS answer: a very weak instrument doesn't fully escape the bias it was supposed to fix.
Try 3: Set the strength back to 0.5 and the direct effect to $0.50. IV is now biased (by about 0.50 ÷ 0.50 = $1). Then lower the strength to 0.2. The same exclusion violation now does much more damage (about 0.50 ÷ 0.20 = $2.50). A weak instrument magnifies any validity problem.

Why weak instruments are so dangerous IV divides by the first stage. If the first stage is around 1, small sample-to-sample wobbles barely matter. If it's around 0.05, the same wobbles can double or flip your estimate. The chapter's phrase for this is estimates that are "big and swingy." A weak instrument also gives you bias: in any real sample the instrument is a little correlated with the hidden stuff by chance, and with a weak first stage that small accident gets blown up.

4. Why this counts as causal

Here is Lecture 11's diagram, with the two things that would ruin the instrument drawn as dashed red arrows.

Education has a back door, education ← ability → wages, and we can't close it because we can't measure ability. IV doesn't close that door. It goes around it. We only use the variation in education that starts at Z. Ability also pushes education around, but that push doesn't start at Z, so it gets thrown away. The book's way to put it: IV doesn't remove the need to close back doors, it moves that job from the treatment to the instrument, where it's hopefully easier. That's why the only arrows that matter are the red ones. Any open path from Z to Y that skips X ruins the estimate.

Arrows into the treatment are fineThe chapter makes a point that surprises many students: other things can cause the treatment (ability → education) without hurting IV, because IV ignores all variation in X that doesn't come from Z. What you need is for Z to have no route to Y except through X, and no shared causes with Y. If some of those routes can be blocked with control variables, the instrument can still be valid after controlling.

5. What has to be true (and how to check it)

Assumption 1: Relevance (the instrument really moves the treatment)

This is the one you can test. Run the first stage and look at the first-stage F-statistic. A bigger F means a stronger instrument, less bias, and less swing.

How big is big enough? There's no single magic number. The old rule of thumb is F > 10, and the chapter calls it very rough. Lecture 11 uses a stricter bar of about 104, which comes from recent research the chapter also cites (confidence intervals are at least a little off until F reaches about 104.7). The chapter's practical advice is to report confidence intervals that stay honest even with weak instruments, such as Anderson-Rubin intervals, rather than just passing or failing an F cutoff.

Assumption 2: Validity / exclusion (no other path from Z to Y)

This is the hard one, and no statistical test can prove it. You have to argue it from your causal diagram and your knowledge of the setting. Ask: "Can I think of any way the instrument affects the outcome other than through the treatment?" Lecture 11 quotes Scott Cunningham's "huh?" test: a good instrument often confuses people when you tell them it's related to the outcome.

Father's education. The chapter notes that parents' education was a popular instrument back in the 1970s and 80s, which it calls "the Wild West." Today most people would doubt it. Educated parents might pass on ability, or help with job connections. Lecture 11 raises the same worry.
Rainfall was a popular instrument for income in farming economies. The chapter describes a study that found rainfall was linked to conflict about equally in areas where dams made rainfall matter little for income. So rainfall must affect conflict some other way, and it fails the exclusion restriction.
Red flag: the same instrument used for many different treatments. If rainfall affects income, migration, and protest, then it has many paths to any one outcome.

Where good instruments come from. The most believable ones involve real randomness. The chapter lists several standard designs: randomly assigned judges (some are harsher) as an instrument for punishment, the draft lottery for military service, compulsory schooling laws for years of education, and first two children of the same sex for having a third child. The cleanest case is a real experiment where not everyone complies. Random assignment is the instrument for actually taking the treatment.

Assumption 3: Monotonicity (no "defiers")

The instrument must push everyone in the same direction (or not at all). The chapter's example is TV ads encouraging exercise. If a few people are so annoyed by the ads that they exercise less, the IV estimate stops being a sensible average. Lecture 12 calls this "ignoring defiers": in a bed-net program, a defier would have to tear down their own nets out of spite because they were offered one. You'll see what defiers do in the next simulator.

6. What you actually estimate: a LATE for compliers

IV only uses people whose treatment is moved by the instrument, so it estimates the effect for those people. Lecture 12 sorts everyone into four types:

CompliersTake the treatment if the instrument says so, and not otherwise.Use a bed net only if the NGO gives them one.

Always-takersTake the treatment no matter what.Already bought their own net.

Never-takersNever take it no matter what.Won't use a net even if handed one.

DefiersDo the opposite of what the instrument says. We usually assume there are none.Offered a net, so they stop using one.

Always-takers and never-takers do the same thing whichever group they're in, so the instrument tells us nothing about their effect. The chapter's exercise-ad example makes the general point: each person's effect is weighted by how much the instrument moves them, and someone the ads don't move at all gets a weight of zero. With a yes/no instrument and a yes/no treatment, every complier moves by the same amount, so IV gives the average effect for compliers. Lecture 12 calls this the CACE (complier average causal effect).

Simulator: who drives the IV estimate?

A made-up population of 100 people, randomly split into two groups. The numbers are invented, chosen to look like Lecture 12's bed-net example. Filled squares use a bed net; hollow squares don't. Health is on a 0–100 scale. In this population, always-takers start out healthier (45) than compliers (40) and never-takers (35).

Always-takersCompliersDefiersNever-takers

Always-takers: 20%

Never-takers: 40%

Defiers: 0% (their effect is 30)

Effect of a net for compliers: 15 points

Effect for always- and never-takers: 5 points

Compliers

—

IV (Wald) estimate

—

Effect for compliers

—

Average effect, everyone

—

Naive: users vs. non-users

—

Try 1: Move the "always- and never-takers" effect slider from 0 to 30. The IV estimate doesn't move at all. IV is blind to people the instrument doesn't push. Meanwhile, the naive comparison of users vs. non-users is off because always-takers were healthier to begin with.
Try 2: Raise always-takers and never-takers so that only 10% are compliers. The IV estimate is still right, but it now rests on very few people. The first stage (the gap in net use) is small, so in real data the estimate would be noisy. This is the weak-instrument problem again.
Try 3: Reset, then add 10% defiers. Defiers benefit more than compliers (30 vs. 15), yet the IV estimate falls below 15. It is no longer an average of anyone's effect. That's why we need the no-defiers (monotonicity) assumption.

Different instruments, different answersBecause IV is about compliers, two valid instruments for the same treatment can give different estimates. Each one moves a different group of people. The chapter's example: if the ads moved some other person more, the estimate would shift toward that person's effect. Always say whose effect you estimated, and don't make claims about the whole population.

Related: fuzzy regression discontinuity is IV with "being past the cutoff" as the instrument. See the RDD reading companion (Lecture 12 calls the result a "doubly local" LATE: compliers, near the cutoff).

7. Doing it in R (with AI help)

In this class, AI writes most of the code. Your job is the design and the checking. These are the steps an IV analysis should include and what to verify at each one:

Step	Typical R	What you check
1. Draw the diagram	(no code: paper, or `dagitty`)	Can you name any path from Z to Y that skips X? Any shared cause of Z and Y? If so, can a control block it?
2. Naive OLS (for comparison)	`feols(wage ~ educ + controls, data = df)`	Which way do you expect the bias to go? (Ability → bias up.)
3. First stage	`feols(educ ~ fathereduc + controls, data = df)`	Is the sign sensible? Is the instrument strong?
4. Reduced form	`feols(wage ~ fathereduc + controls, data = df)`	Same sign as you'd expect? Reduced form ÷ first stage should roughly equal the IV estimate.
5. 2SLS in one command	`feols(wage ~ controls \| educ ~ fathereduc, data = df)` (fixest) or `iv_robust(wage ~ educ + controls \| fathereduc + controls, data = df)` (estimatr)	Are the controls in both stages? (Both commands handle this if you write them as shown.) Did the AI put `educ` (instrumented) and `fathereduc` (instrument) in the right slots, not swapped?
6. Instrument strength	`fitstat(m, "ivf")` or `summary(m)` (fixest); `iv_robust(..., diagnostics = TRUE)`	What is the first-stage F? Weak-instrument-robust (Anderson-Rubin) intervals, e.g. from the `ivDiag` package, if F isn't large.
7. Interpret	—	Who are the compliers? Say "the effect for people whose education was moved by their father's education," not "for everyone."

Don't do 2SLS by hand for your final answerRunning the first stage, saving the predicted values, and then running a second lm() gives the right estimate (Lecture 11 shows this) but the wrong standard errors, because the second stage treats the predicted values as if they were real data. Both the chapter and the lecture say to use a dedicated IV command.

Common mistakes

Claiming a strong first stage proves the instrument is valid. Relevance and validity are separate. Only relevance can be tested.
"Testing" validity by adding the instrument as a control in the outcome regression. The chapter explains this is collider bias (you're controlling for X, which both Z and the hidden stuff cause), so it tells you nothing.
Treating a non-significant overidentification or Hausman test as proof of validity. Failing to find a problem isn't the same as showing there is none.
Putting a control in the second stage but not the first.
Using a probit or logit first stage and plugging its predictions into the second stage. The chapter calls this the "forbidden regression." (That's a different thing from the "forbidden" column in Lecture 11, which controlled for ability we can't actually measure.)
Reporting the IV estimate as the effect for everyone (it's a LATE for compliers).
Ignoring a weak first stage, or dropping a project only because F fell under 10. The chapter argues that F-cutoff habits leave published work biased. Honest intervals are better.

8. How to read the chapter

The chapter is long (roughly 15,000 words) and parts of it are written for graduate students. Here's where to spend your time:

Section	What it covers
19.1.1 Isolating Variation	The statue analogy, IV as a stand-in for an experiment, and a six-panel picture of IV with a yes/no instrument. Study that figure.	Read closely
19.1.2 Assumptions	Relevance, validity, the path-by-path diagram exercise, and the rainfall and health/growth cautionary tales.	Read closely
19.1.3 Canonical Designs	Judges, the draft lottery, compulsory schooling, shift-share. Good for ideas.	Skim
19.2.1 IV Estimators	2SLS (read this part), then GMM and overidentification (skim). The insurance-defaults example shows a real first and second stage.	Skim
19.2.2 IV and Treatment Effects	Why IV is a weighted LATE, compliers and defiers, and monotonicity.	Read
19.2.3 Checking the Assumptions	Weak instruments and the F-test (read). The validity tests and why they don't prove much (skim).	Read
19.3.1–19.3.4 How the Pros Do It	Weak-instrument fixes, beyond-LATE methods, nonlinear IV, partially invalid instruments. Beyond this course.	Optional