Why Random, and the Limits of One Sample

Lesson 2 of 2: Bias and Variability

In this lesson:

• See why a non-random sample gives a biased answer
• Understand why one sample is never certain

A Sample Collected Badly on Purpose

Same question: what fraction of Lincoln High would re-enroll?

This time we survey only students leaving the elective's classroom.

• These are the people who already chose it
• We ask them, and compute

Something should already feel wrong. What is it?

Predict the Answer Before You Calculate

The true fraction across all 2,400 is .

Surveying only students leaving the elective, our will be:

• A. Too high B. Too low C. About right

Commit to one. Then ask yourself why.

Bias Is a Systematic Tilt

The students leaving the elective already enrolled — they lean toward yes.

• Our is pushed high, every time
• This is bias: a systematic error, not random noise

Noise scatters and averages out. Bias leans one way — and never corrects.

Three Ways Sampling Goes Wrong

• Selection: the choosing method favors some
• Voluntary-response: only the passionate reply
• Undercoverage: whole groups can't be chosen

What a Random Sample Requires

A random sample uses a real chance mechanism:

• Every member has a known, equal chance of selection
• Lottery, random-number draw, names from a hat

Everyday "random" means haphazard. Statistically, "grab whoever" is the opposite of random.

Random or Biased? You Decide

A TV show asks viewers to text in their vote; thousands respond.

• Random sample of viewers — yes or no?
• If biased, which kind?

Think about who chooses to text, and who never gets counted.

1936: A Poll of Millions, Wrong

• Literary Digest: 2+ million responses → wrong
• Gallup: a few thousand, random → right

Size Shrinks Noise, Not Bias

The Digest's list skewed wealthy — biased. Two million biased responses are still biased.

• Randomness removes bias
• Size only reduces variability (random scatter)

Ask "how were they chosen?" before "how many?"

Classify Each of These Methods

Random or biased? Name the bias and its direction:

1. Computer randomly selects 200 from a full patient list
2. A website counts whoever clicks its poll
3. A reporter interviews people at one coffee shop

Decide all three before advancing.

Bias Is Fixed — But It Still Varies

Use a random sample and bias is gone.

But two different random samples won't match.

• Each sample is a different slice of the population
• Different slices → different statistics

Randomness removes bias. It does not remove variability.

Three Random Samples, Three Answers

• Same population, three random samples of 60
• = 0.62, 0.68, 0.65 — all different

The Parameter Holds; the Statistic Moves

This sample-to-sample change is sampling variability.

• The parameter stayed fixed the whole time
• Only the statistic moved

A single statistic estimates the parameter — it is never the exact parameter.

Could the Statistic Be the Parameter?

Three honest samples gave 0.62, 0.68, 0.65.

If truly equaled —

• how could three samples give three different values?

They can't all be the one fixed truth. So what must really be?

Larger Samples Bounce Around Less

• Small samples spread wide; large samples cluster tight
• Size buys precision, not freedom from bias

What One Sample Can and Can't Do

A single random sample can:

• give a best estimate, with a sense of its precision

It cannot:

• pin the parameter exactly, or certify a conclusion

The honest answer is "around 0.65, give or take" — never "0.65, period."

Reason About a Real Headline

A random poll finds 52% support. The headline reads: "52% of voters support the measure."

• Would a different random sample give exactly 52%?
• Is the headline honest? What should it say?

Write your reasoning before advancing.

Three Common Errors to Avoid

"Bigger is better" — not if it's biased; size never fixes bias
" is " — it's an estimate; three samples gave three values
"Random means certain" — randomness removes bias, not variability

Each error has its own fix. Spotting them is reading numbers wisely.

Key Takeaways From Lesson Two

✓ Random removes bias — estimate centered on the truth
✓ Each sample is a slice, so the statistic still varies
✓ Larger samples shrink the give-or-take, never to zero

Size never fixes bias; randomness never gives certainty

Coming Up Next: Surprising Results

In the next lesson, you'll learn to measure how surprising a result is.

If a model were true, how often would it produce data this extreme? That question drives the rest of the unit.