Unit 4 · Topic 4.8 Beta

Justifying a Claim Based on a Confidence Interval for the Difference Between Two Population Means

Use an interval for μ_1 − μ_2 to judge claims: if 0 is outside, there is convincing evidence of a difference in the direction of the interval's sign; if 0 is inside, there is not, and no difference is proven.

Practice 4: Interpret Results

Part 1 · Hook

Why this matters

Students who wore noise-canceling headphones scored 4.5 points higher on a reading test than students who did not. The school will only buy headphones if they add more than 5 points. One interval answers two questions: is there any real difference, and is it big enough to matter?

Part 2 · Before you start

What this builds on

Part 3 · Prerequisite check

Quick check before you start

1. A 95% interval for μ1−μ2\mu_1 - \mu_2 is from −1.2-1.2 to 3.4. Which is true?

  1. 0 is a plausible value for the difference
  2. The interval proves μ1=μ2\mu_1 = \mu_2
  3. Group 1 has the larger mean
  4. The interval is invalid because it has a negative end
Show the answer

0 is inside the interval, so no difference is plausible, as are differences in either direction.

  • Correct: 0 is a plausible value for the difference:
  • The interval proves μ1=μ2\mu_1 = \mu_2:
  • Group 1 has the larger mean:
  • The interval is invalid because it has a negative end:

2. For two proportions, an interval for p1−p2p_1 - p_2 entirely above 0 gives

  1. convincing evidence that p1>p2p_1 > p_2
  2. proof that p1>p2p_1 > p_2
  3. no evidence either way
  4. evidence that p1<p2p_1 < p_2
Show the answer

Every plausible difference is positive, so the evidence points to p1>p2p_1 > p_2.

  • Correct: convincing evidence that p1>p2p_1 > p_2:
  • proof that p1>p2p_1 > p_2:
  • no evidence either way:
  • evidence that p1<p2p_1 < p_2:

Part 4 · See it

See it first

Three confidence intervals for the true difference in mean reading score, headphones minus no headphones, each centered at 4.5 points: 90% from 1.35 to 7.65, 95% from 0.72 to 8.28 and 99% from −0.53 to 9.53. A dashed line at 0 lies left of the 90% and 95% intervals but inside the 99% interval. A second dashed line at 5 points, the school's threshold, lies inside all three.
90%, 95% and 99% intervals for the headphones effect. 0 is outside the 90% and 95% intervals but inside the 99% interval. The 5-point threshold is inside all three, so no interval shows the effect is large enough for the school. LevlPrep original diagram.

Part 5 · Step by step

How it works, step by step

  1. Look at where 0 sits in the interval for μ1−μ2\mu_1 - \mu_2outside: convincing evidence of a difference (its sign gives the direction); inside: no convincing evidence
  2. Compare the interval with the smallest difference that matters in contextthat judges practical significance, which a p-value or "0 is outside" cannot
  3. Ask how the data were producedrandom assignment supports cause and effect; random sampling supports generalizing
  4. 0 inside the intervalis never proof of no difference: the plausible values still range from one direction to the other

Part 6 · Key ideas

Key ideas

  • 0 outside the interval: "Because every value in the interval is greater than 0, the interval gives convincing evidence that the true mean ... is greater for group 1 than for group 2."
  • 0 inside the interval: "Because 0 is in the interval, the interval does not give convincing evidence that the true means differ." Never "the means are equal".
  • Practical significance: a difference can be statistically significant yet too small to matter; compare the plausible values with a threshold from the context.
  • Reversing the order of subtraction flips the interval's signs, not the conclusion.

Part 7 · Misconception

A common mistake

The wrong idea: 0 is outside the interval, so the difference is large and important.

What actually happens: Excluding 0 shows a difference exists (statistical significance). Whether it is big enough to matter (practical significance) depends on the plausible values compared with what matters in context; with very large samples, a tiny difference can exclude 0.

Part 8 · Check yourself

Check yourself

Exam-style questions. Anything you miss goes into your review queue.

Table

Noise-canceling headphones and reading scores

Sixty students at a large high school volunteered for a study. They were randomly assigned, 30 to each group, to take a timed reading test (out of 100) with noise-canceling headphones (H) or without them (N), in a noisy room (invented data). Boxplots of both groups show no strong skew or outliers. The school would buy headphones only if they raise the mean score by more than 5 points.

Summary of reading scores and a 95% two-sample t interval for μH−μN\mu_H - \mu_N
GroupnMeanSD
Headphones (H)3074.68.1
No headphones (N)3070.16.4
95% interval for μH−μN\mu_H - \mu_Nfrom 0.72 to 8.28 points

1. Does the interval give convincing evidence that headphones raise the mean reading score? Build the justification.

Write the conclusion in context.

Show the answer

The claim is μH−μN>0\mu_H - \mu_N > 0. Every plausible value in the interval is positive, so 0 (no difference) is not plausible.

  • A full-credit answer: Because every value in the interval from 0.72 to 8.28 is greater than 0, the interval gives convincing evidence that the true mean reading score is higher with headphones than without, for students like these.

2. Can the school conclude that the headphones caused the higher scores?

  1. No, because the students who took part were volunteers, not a random sample
  2. No, since a confidence interval by itself is unable to show cause and effect
  3. Yes, for students like these volunteers, because of random assignment
  4. Yes, for each high school student in the country, since the groups were random
Show the answer

Random assignment allows a cause-and-effect conclusion; random sampling (absent here) would allow generalizing to a larger population.

  • No, because the students who took part were volunteers, not a random sample: Volunteers limit how far the results generalize, not the causal conclusion for students like them.
  • No, since a confidence interval by itself is unable to show cause and effect: Cause and effect comes from the design. With random assignment, an interval can support a causal claim.
  • Correct: Yes, for students like these volunteers, because of random assignment: Random assignment makes the groups alike on average except for the headphones, so the difference can be credited to them. The volunteers were not a random sample, so generalizing to all students is not justified.
  • Yes, for each high school student in the country, since the groups were random: The volunteers were not a random sample of all students.

3. Does the interval give convincing evidence that headphones raise the mean score by more than 5 points, the school's threshold?

  1. No, because values from 0.72 to 5 are inside the interval
  2. Yes, because the sample difference of 4.5 points is close to the threshold of 5
  3. Yes, because each value in the interval is greater than 0
  4. No, because the interval proves the gain is less than 5
Show the answer

A difference can be statistically significant (0 outside the interval) without being shown practically important (more than 5 points).

  • Correct: No, because values from 0.72 to 5 are inside the interval: Gains smaller than 5 points are plausible, so the evidence does not show the gain is large enough to matter to the school.
  • Yes, because the sample difference of 4.5 points is close to the threshold of 5: The sample difference is below 5, and smaller true gains are plausible.
  • Yes, because each value in the interval is greater than 0: That supports some gain, not a gain of more than 5.
  • No, because the interval proves the gain is less than 5: Values above 5, up to 8.28, are also plausible; nothing is proven.

4. A second analyst subtracts in the other order and reports an interval for μN−μH\mu_N - \mu_H. What interval and conclusion does she get?

  1. From 0.72 to 8.28, so students without headphones score higher on average
  2. From −8.28-8.28 to −0.72-0.72, with the same conclusion
  3. From −0.72-0.72 to −8.28-8.28, so there is no convincing evidence of a difference
  4. From 8.28 to 0.72, and the conclusion now depends on the order used
Show the answer

If 0.72<μH−μN<8.280.72 < \mu_H - \mu_N < 8.28, then −8.28<μN−μH<−0.72-8.28 < \mu_N - \mu_H < -0.72.

  • From 0.72 to 8.28, so students without headphones score higher on average: The parameter changed, so the signs must change too.
  • Correct: From −8.28-8.28 to −0.72-0.72, with the same conclusion: Reversing the order flips the signs. Every value is now below 0, which says the no-headphones mean is lower: the same finding.
  • From −0.72-0.72 to −8.28-8.28, so there is no convincing evidence of a difference: The ends must be in order (lower first), and 0 is still outside.
  • From 8.28 to 0.72, and the conclusion now depends on the order used: The conclusion does not depend on which group is subtracted, and the lower end comes first.

5. Two huge random samples of 20,000 phones each give a 95% interval for the difference in mean battery life (brand X minus brand Y) from 0.03 to 0.09 hours. Which statement is the best summary?

  1. The difference is statistically significant but too small to matter
  2. The difference is large, because 0 is not in the interval
  3. There is no difference, because the interval is near 0
  4. The interval is unreliable because the samples are far too large to trust
Show the answer

Practical significance asks whether the plausible differences are big enough to matter in context. Here the largest plausible difference is 0.09 hours, about 5 minutes.

  • Correct: The difference is statistically significant but too small to matter: Every value is above 0, so X lasts longer on average, but by at most about 5 minutes: statistically significant, likely not practically significant.
  • The difference is large, because 0 is not in the interval: Excluding 0 shows there is a difference, not that it is large.
  • There is no difference, because the interval is near 0: The interval excludes 0, so there is convincing evidence of a difference.
  • The interval is unreliable because the samples are far too large to trust: Large samples make intervals precise; that is why a tiny difference is detected.

6. In a randomized experiment, the 95% interval for the true mean weight loss on diet A minus diet B is from 0.4 to 1.1 kilograms over a year. A doctor considers 3 kilograms the smallest change worth a patient's effort. Which statement fits?

  1. Diet A works better on average, but not by an important amount
  2. The diets are equally effective, because the interval is close to 0
  3. Diet A is clearly worth recommending, because 0 is outside the interval
  4. Nothing can be said, since 3 is outside the interval
Show the answer

Statistical significance: 0 is outside. Practical significance: the plausible differences, at most 1.1 kilograms, are well below 3.

  • Correct: Diet A works better on average, but not by an important amount: Every value is above 0, so diet A gives more loss on average; every value is also far below 3 kilograms.
  • The diets are equally effective, because the interval is close to 0: 0 is outside the interval, so there is convincing evidence of a difference.
  • Diet A is clearly worth recommending, because 0 is outside the interval: Statistical significance alone does not make the difference large enough to matter.
  • Nothing can be said, since 3 is outside the interval: Knowing 3 is outside is useful: gains that large are not plausible.

Part 9 · Summary

Summary

Use an interval for μ1−μ2\mu_1 - \mu_2 to judge claims: if 0 is outside, there is convincing evidence of a difference in the direction of the interval's sign; if 0 is inside, there is not, and no difference is proven. Then judge practical significance by comparing the plausible values with what matters in context, and judge cause and generalization from the design.

Part 10 · Up next

What comes next

Part 11 · Connections

Connections

Share to Google Classroom (opens in a new tab)

Question set for this topic