Three Study Types and the Two Jobs of Randomness

Lesson 1 of 2: Designs and Randomness

In this lesson:

• Distinguish surveys, observational studies, and experiments
• See the two different jobs the word "random" does

Learning Objectives for This Unit

By the end of this two-lesson unit, you should be able to:

1. Distinguish surveys, observational studies, experiments
2. Identify explanatory, response, treatment, and control
3. Explain random selection versus random assignment
4. Explain why observation can't prove causation
5. Classify a study and state its conclusion

One Tutoring Question, Three Answers

Does a new after-school tutoring program raise test scores?

We'll attack this one question three different ways:

• a survey, an observational study, an experiment

The question stays fixed. Only the method changes — so the design decides the answer.

The Sample Survey: Measure and Report

A sample survey selects students (ideally at random) and asks or measures.

• Reports a characteristic of the population
• The researcher does not intervene

A survey takes a snapshot. Nobody is put into a group; nothing is done to anyone.

The Observational Study: Watch Existing Groups

An observational study compares groups that already differ.

• Tutoring-choosers vs. non-choosers — they sorted themselves
• The researcher still does not intervene

We compare pre-existing groups. But they might differ in other ways too.

The Experiment: Impose a Treatment

• An experiment randomly assigns who gets the treatment
• The researcher imposes it, rather than watching

Compare the Three Designs Side by Side

The bottom row — what each can conclude — is fixed by the design, not by care or size.

The Vocabulary the Cluster Reuses

• Treatment: the thing imposed (tutoring)
• Control group: the baseline — not wasted subjects
• Explanatory variable: tutoring or not (the suspected cause)
• Response variable: the test score (the outcome)

Four terms you'll see in every study from here on.

Classify This Study and Name Why

A researcher records whether 500 adults exercise and their heart rate, then compares the groups.

• Survey, observational study, or experiment?
• What structural feature decides it?

Did the researcher impose anything, or just measure? Commit before advancing.

One Word "Random," Two Different Jobs

That was an observational study — no one was assigned.

Notice: the survey selected randomly; the experiment assigned randomly.

• Same word — two different jobs

Separating these two jobs is the key idea of the whole cluster.

Random Selection: Who Gets Into the Study

Random selection draws the sample at random from the population.

• The sample fairly represents the whole
• Payoff: results generalize to the population

Selection chooses who is studied.

Random Assignment: What Each Subject Gets

Random assignment uses chance to place subjects into treatment or control.

• The two groups start out alike on everything
• Payoff: a difference can be attributed to the treatment

Assignment chooses what each subject gets.

Selection Earns the Right to Generalize

Random selection is what earns the right to generalize.

• A random sample is a fair miniature of the population
• So a finding in the sample extends to the population

No random selection → no generalizing beyond the people studied.

Assignment Earns the Right to Cause

Random assignment is what earns the right to claim cause.

• Chance balances the two groups on everything at the start
• So an outcome difference must be the treatment

No random assignment → no causal claim, only association.

The 2x2: Crossing the Two Randomizations

Selection governs generalization; assignment governs causation. Independent, then combined.

Two Arrows: Population to Sample to Groups

• Arrow 1 (selection) earns generalization; Arrow 2 (assignment) earns causation

Which Randomization Did This Study Use?

200 volunteers are randomly assigned to a new drug or a placebo.

• Selection, assignment, both, or neither?

Were they drawn from a population, or did they volunteer? Were they randomly grouped? Commit first.

Two Studies: What Does Each License?

• A. Randomly selects 1,000 voters, asks their choice
• B. 50 volunteers randomly assigned to two diets

For each: can it generalize? claim cause? both? neither?

Work out the conclusion for each — they're opposite cases.

Key Takeaways From Lesson One

✓ Survey measures; observational watches; experiment imposes
✓ Selection earns generalization; assignment earns causation
✓ The two randomizations are independent — either, both, neither

Never collapse selection and assignment into one "random?"
The control group is the baseline, not wasted subjects

Coming Up Next: The Hidden Variable

In Lesson 2, you'll find out why an observational study can't prove cause.

The answer is a hidden third variable — a confounder — and learning to spot it is the real skill.