The Design Lens and the Verdict

Lesson 2 of 2: Matching Claim to Design

In this lesson:

• Match a claim's verb to its study design and name overreach
• Weigh all four lenses into one calibrated verdict

Self-Selected Coffee — Can It Prove Cause?

In the coffee study, students chose whether to drink coffee. Nobody assigned them.

• The two groups differ in countless ways besides coffee
• Can a study where people sort themselves prove coffee causes scores?

You sense the answer is no. The design lens says why.

It's Observational — and Here's a Confounder

A hidden third variable drives both — random assignment would have balanced it.

Self-Selection Can License Association Only

A self-formed group can show only that two things travel together.

• We can say coffee-drinking and scores are associated
• We cannot say coffee caused the scores — a confounder could

B.3: observational = association; only experiments license cause.

The Causal Verb Overreaches the Design

The headline says scores rise because of coffee — a causal claim.

• The design licenses only association
• Claim strength outruns design power → causal overreach

The number isn't a lie — the causal conclusion is unearned.

The Verb-vs-Design Matching Routine, Step by Step

Run this on every headline. A causal verb on a non-experiment is overreach.

Read the Verb: Causal vs Associational

• Causal: causes, raises, improves, boosts, leads to — asserts one made the other
• Associational: linked to, associated with, correlated with — only travels together

Media use causal verbs for associational findings — read the actual word.

Check the Design: Was Anything Assigned?

The one feature that licenses cause is random assignment.

• Not random selection (who's in the sample) — random assignment (who's in which group)
• Subjects chose their group, or groups pre-existed → no assignment

Ask bluntly: was anything randomly assigned?

"Treatment Works"? Demand Assignment + Significance

For a real treatment claim, demand the B.5 machinery:

• Was there random assignment — is a causal reading even possible?
• Was the difference checked for significance, or is it a raw gap?

A gap with no significance check hasn't ruled out luck.

A "Linked-To" Survey Headline — Flag It?

"People who eat breakfast are linked to lower body weight" — from a self-report survey.

• What's the verb? Was anything assigned? Does the claim overreach?
• Name a plausible confounder

The verb is honest here — does that change your verdict?

Match Verb to Design; Name the Overreach

"Teens who play a musical instrument score higher on math tests."

• Classify the verb; was anything assigned; name the overreach
• Name the missing design feature, and a plausible confounder

Write the full routine before advancing.

Two Lenses Left, Then Weigh Them All

The design lens caught the causal overreach. Two lenses remain:

• Uncertainty: is a sample size, margin of error, or significance given?
• Display: is the graph honest, or engineered to mislead?

Then the capstone: weigh all four into one verdict.

Is the Report's Uncertainty Disclosed?

Look for three things: sample size, margin of error, significance.

• A bare "15% higher" with no and no significance is weak
• You can't separate signal from noise without them

A.2, B.4, B.5: a report that hides uncertainty hides what you'd judge it by.

Same Data, Two Axes — Why It Changes

Read the axis before the bars — same data, opposite impression.

Other Graph Tricks to Watch For

Inconsistent or stretched scales — distort comparisons
Cherry-picked time windows — show only the supportive slice
Missing axis labels — let the chart imply anything

Every trick manipulates the frame around honest numbers.

Weigh the Evidence, Don't Gotcha

Evaluation is not one flaw = "fake" or one strength = "proven."

• Reports are mixtures of strengths and weaknesses
• One flaw needn't condemn; one strength doesn't vindicate

The skill is calibration — how much to believe, not a binary stamp.

The Calibrated Four-Lens Coffee Verdict

Sampling: undisclosed, likely small and self-selected → may not generalize
Design: observational, so "because of coffee" overreaches — confounder likely
Uncertainty: no , no significance → the 15% is unanchored
Display: truncated axis exaggerates the gap

Verdict: the association may be real, but the causal headline is unsupported.

Write a Full Four-Lens Verdict

A vitamin site shows a y-axis-at-40 chart claiming users boosted memory 20%, from 50 volunteers.

• Run all four lenses — record one specific finding each
• Synthesize a calibrated verdict: how much to believe?

No scaffolding. This is the exit task of the whole domain.

Four Errors in Reading Data Reports

"Linked to" = "causes" — no, ask what was randomly assigned
Numbers + "study" = trustworthy — no, numbers ≠ good data
A graph can't lie — no, read the axis first
Evaluation = one gotcha — no, weigh strengths and weaknesses

Key Takeaways From Lesson Two

✓ Match verb to design — causal verb + observational = overreach
✓ Check disclosed uncertainty: , margin of error, significance
✓ Weigh all four lenses into a calibrated verdict

Association is not cause without random assignment
The same number can mislead — read the axis first

Domain Wrap-Up: The Habit You Now Hold

This closes the IC domain — and it's all one habit now.

Sampling, simulation, study design, margin of error, significance, and the evaluation lenses let you take any data claim apart and say, with reasons, how much to believe.