Exercises: Compare Two Treatments Using Re-Randomization Simulation
Work through each section in order. Show your work where indicated. For each significance decision, state the tail fraction and whether the result is significant, then interpret it.
Recall / Warm-Up
In a randomized experiment, the treatment group's mean improvement is points and the control group's mean improvement is points. What is the observed difference , in points?
To test whether a treatment had a real effect, we begin by assuming a model. What is the model we assume and test?
The treatment has no effect, so any observed difference is due only to the random assignment.
The treatment definitely has an effect equal to the observed difference.
The two groups had exactly equal outcomes.
The experiment was not randomized.
In a drug trial, of the treatment group recovered and of the control group recovered. What is the observed difference in recovery proportions ? Give your answer as a decimal.
Fluency Practice
A randomized experiment finds that the treatment group's mean is 3 points higher than the control group's. A student concludes, "The treatment worked." Why is this conclusion premature?
Because even with no effect, random assignment alone routinely produces gaps of a few points; we must first check whether a 3-point gap is bigger than chance usually produces.
Because the treatment group should always score lower than the control group.
Because a difference of 3 points is too small to ever matter.
Because means cannot be compared between two groups.
A treatment group of 5 subjects has improvement scores 10, 12, 14, 8, 11. A control group of 5 subjects has scores 9, 7, 10, 8, 6. Compute the observed difference in means , in points.
During one re-randomization shuffle, which of the following actually moves, and which stays fixed?
The T/C group labels are reshuffled; each subject keeps its own outcome value.
The outcome values are reshuffled; the group labels stay fixed.
Both the outcome values and the labels are randomly changed.
Nothing moves; we simply recompute the same difference.
The dot plot shows 50 re-randomization differences built under the no-effect model. The observed difference is +4 points. (a) How many of the 50 shuffles produced a difference of +4 or more? (b) What fraction of shuffles is that (as a decimal)?
Using the same dot plot, the observed difference of +4 lands in the far upper tail, with only about 4% of shuffles at least that large. What is the significance decision?
Significant: a gap this large is rarely produced by chance alone, so it is evidence the treatment had a real effect.
Not significant: 4% of shuffles still produced such a gap, so it is plainly chance.
Impossible to decide without a formal p-value formula.
Significant, but only if the distribution were centered at +4.
You're viewing 2 of 6 sections.
Create a free account to continue the full exercise set and save your progress.
Create free account