In this lesson:
By the end of this two-lesson unit, you should be able to:
Does a new after-school tutoring program raise test scores?
We'll attack this one question three different ways:
The question stays fixed. Only the method changes — so the design decides the answer.
A sample survey selects students (ideally at random) and asks or measures.
A survey takes a snapshot. Nobody is put into a group; nothing is done to anyone.
An observational study compares groups that already differ.
We compare pre-existing groups. But they might differ in other ways too.
The bottom row — what each can conclude — is fixed by the design, not by care or size.
Four terms you'll see in every study from here on.
A researcher records whether 500 adults exercise and their heart rate, then compares the groups.
Did the researcher impose anything, or just measure? Commit before advancing.
That was an observational study — no one was assigned.
Notice: the survey selected randomly; the experiment assigned randomly.
Separating these two jobs is the key idea of the whole cluster.
Random selection draws the sample at random from the population.
Selection chooses who is studied.
Random assignment uses chance to place subjects into treatment or control.
Assignment chooses what each subject gets.
Random selection is what earns the right to generalize.
No random selection → no generalizing beyond the people studied.
Random assignment is what earns the right to claim cause.
No random assignment → no causal claim, only association.
Selection governs generalization; assignment governs causation. Independent, then combined.
200 volunteers are randomly assigned to a new drug or a placebo.
Were they drawn from a population, or did they volunteer? Were they randomly grouped? Commit first.
For each: can it generalize? claim cause? both? neither?
Work out the conclusion for each — they're opposite cases.
✓ Survey measures; observational watches; experiment imposes ✓ Selection earns generalization; assignment earns causation ✓ The two randomizations are independent — either, both, neither
Never collapse selection and assignment into one "random?" The control group is the baseline, not wasted subjects
In Lesson 2, you'll find out why an observational study can't prove cause.
The answer is a hidden third variable — a confounder — and learning to spot it is the real skill.
Click to begin the narrated lesson
Distinguish surveys, experiments, and observational studies