Unit 3 · Topic 3.8 Beta

Potential Errors When Performing Tests

A Type I error rejects a true H_0 (probability α ); a Type II error fails to reject a false H_0 (probability β , against a specific alternative).

Practice 2: Collect DataPractice 4: Interpret Results

Part 1 · Hook

Why this matters

A city will replace its old water pipes only if a test finds convincing evidence that more than 10% of homes have too much lead. The test can go wrong in two ways: the city spends millions on pipes it did not need, or it leaves families drinking unsafe water. Which mistake is worse, and how can the city make it less likely?

Part 2 · Before you start

What this builds on

Part 3 · Prerequisite check

Quick check before you start

1. A test at $\alpha = 0.05$ gives a p-value of 0.03. What is the decision?

Reject $H_0$ , because $0.03 < 0.05$
Fail to reject $H_0$ , because $0.03 < 0.05$
Accept $H_0$ , because 0.03 is small
Reject $H_a$ , because $0.03 < 0.05$

Show the answer

Reject $H_0$ when the p-value is less than $\alpha$ .

Correct: Reject $H_0$ , because $0.03 < 0.05$ :
Fail to reject $H_0$ , because $0.03 < 0.05$ :
Accept $H_0$ , because 0.03 is small:
Reject $H_a$ , because $0.03 < 0.05$ :

2. If $\hat{p}$ is approximately normal with mean 0.60 and standard deviation 0.0346, which expression gives $P(\hat{p} \ge 0.558)$ ?

$P\left(Z \ge \frac{0.558 - 0.60}{0.0346}\right)$
$P\left(Z \le \frac{0.558 - 0.60}{0.0346}\right)$
$P\left(Z \ge \frac{0.60 - 0.558}{0.0346}\right)$
$P(Z \ge 0.558)$

Show the answer

Standardize the boundary with the distribution's own mean and standard deviation, and keep the direction.

Correct: $P\left(Z \ge \frac{0.558 - 0.60}{0.0346}\right)$ :
$P\left(Z \le \frac{0.558 - 0.60}{0.0346}\right)$ :
$P\left(Z \ge \frac{0.60 - 0.558}{0.0346}\right)$ :
$P(Z \ge 0.558)$ :

Part 4 · See it

See it first

Two normal curves for the sample proportion from samples of 200. The left curve, centered at the null value 0.50, is the distribution if the null is true; the right curve, centered at 0.60, is the distribution if the true proportion is 0.60. A vertical line at the cutoff 0.558 marks where the test rejects H0 at alpha = 0.05 (sample proportions above it). Under the null curve, the small area beyond the cutoff is alpha, the probability of a Type I error. Under the alternative curve, the area to the left of the cutoff is beta, the probability of a Type II error, about 0.11; the area to the right is the power, about 0.89. — The null distribution (p = 0.5) and one alternative (p = 0.60) for samples of 200, with the rejection cutoff: α under the null curve, β and power under the alternative. LevlPrep original diagram.

Part 5 · Step by step

How it works, step by step

$H_0$ is true but the sample happens to look extremethe test rejects $H_0$ : a Type I error, with probability $\alpha$
$H_a$ is true but the sample happens to look ordinarythe test fails to reject $H_0$ : a Type II error, with probability $\beta$
A specific alternative value is truethe test rejects $H_0$ with probability power $= 1 - \beta$
The sample size grows (or $\alpha$ grows, or the true value is farther from $p_0$ )the curves overlap less or the cutoff is easier to pass, so power rises and $\beta$ falls

Part 6 · Key ideas

Key ideas

Type I: $H_0$ true, but we reject it (a false alarm). Probability $\alpha$ .
Type II: $H_a$ true, but we fail to reject $H_0$ (a missed effect). Probability $\beta$ , which depends on the true value.
Describe an error with the condition first: "if the true proportion of all __ is __, but we find (do not find) convincing evidence that ...", then the consequence in context.
Power $= 1 - \beta$ : the probability of rejecting $H_0$ when a specific alternative is true.
More power: larger $n$ (best), larger $\alpha$ , or a true value farther from $p_0$ . For a fixed $n$ , lowering $\alpha$ raises $\beta$ .

Part 7 · Misconception

A common mistake

The wrong idea: A Type I error is when the test gets the answer wrong, and a Type II error is when it gets it wrong a second time.

What actually happens: Each is a specific mistake: Type I rejects a true $H_0$ ; Type II fails to reject a false $H_0$ . After a rejection only a Type I error is possible; after a failure to reject, only a Type II error.

Part 8 · Check yourself

Check yourself

Exam-style questions. Anything you miss goes into your review queue.

Table

Lead in household water

A city must replace its old water pipes if more than 10% of its homes have lead levels above a safety limit. The water department tests a random sample of 300 of the city's 40,000 homes and will test $H_0: p = 0.10$ against $H_a: p > 0.10$ , where $p$ is the true proportion of all the city's homes with lead above the limit. If the test finds convincing evidence for $H_a$ , the city replaces the pipes.

The four possible outcomes of the test
	$H_0$ is true ( $p = 0.10$ )	$H_a$ is true ( $p > 0.10$ )
Reject $H_0$	Type I error	Correct decision
Fail to reject $H_0$	Correct decision	Type II error

1. Which describes a Type I error in this context?

The proportion is 0.10, but the test finds convincing evidence that it is greater
The proportion is greater than 0.10, but the test does not find convincing evidence
The proportion is greater than 0.10, and the test finds convincing evidence of that
The sample proportion turns out to be greater than 0.10

Show the answer

A Type I error is rejecting a true $H_0$ : the true proportion is 0.10, but the test finds convincing evidence that it is greater than 0.10.

Correct: The proportion is 0.10, but the test finds convincing evidence that it is greater: Right: $H_0$ is true and we reject it.
The proportion is greater than 0.10, but the test does not find convincing evidence: That is a Type II error: $H_a$ is true and we fail to reject $H_0$ .
The proportion is greater than 0.10, and the test finds convincing evidence of that: That is a correct decision.
The sample proportion turns out to be greater than 0.10: An error is about the decision compared with the truth, not about the sample result.

2. Which describes a Type II error in this context?

The proportion is 0.10, and the test finds convincing evidence that it is greater
Over 10% of homes are above the limit, but the test finds no convincing evidence
The test fails to reject $H_0$ when exactly 10% of homes are above the limit
The department tests too few homes to find anything

Show the answer

A Type II error is failing to reject a false $H_0$ : more than 10% of homes really are above the limit, but the test does not find convincing evidence of it.

The proportion is 0.10, and the test finds convincing evidence that it is greater: That is a Type I error.
Correct: Over 10% of homes are above the limit, but the test finds no convincing evidence: Right: $H_a$ is true and we fail to reject $H_0$ .
The test fails to reject $H_0$ when exactly 10% of homes are above the limit: If $H_0$ is true, failing to reject it is the correct decision.
The department tests too few homes to find anything: A small sample can make a Type II error more likely, but the error itself is a wrong decision about $H_0$ .

3. What is a consequence of a Type II error here?

The city spends money replacing pipes that did not need replacing
The city replaces the pipes, as it should
The city does not replace the pipes, and residents keep drinking unsafe water
The city has to test a second random sample

Show the answer

After a Type II error the city does not replace the pipes, even though more than 10% of homes have lead above the limit. Residents keep drinking water with too much lead.

The city spends money replacing pipes that did not need replacing: That is the consequence of a Type I error.
The city replaces the pipes, as it should: That follows a correct decision to reject $H_0$ when $H_a$ is true.
Correct: The city does not replace the pipes, and residents keep drinking unsafe water: Right: the problem exists but the test misses it.
The city has to test a second random sample: Nothing in the plan calls for a second sample; the decision is made from this test.

4. Build the description of a Type I error and its consequence in this context.

Write the conclusion in context.

Show the answer

State the condition first (what is true), then the wrong decision, then what happens because of it.

A full-credit answer: A Type I error would occur if the true proportion of all the city's homes with lead above the limit is 0.10, but the test finds convincing evidence that it is greater than 0.10. The city would then spend money replacing pipes that did not need replacing.

Graph

Will customers prefer the new recipe?

A snack company will switch to a new recipe only if there is convincing evidence that more than half of its customers prefer it. It plans a one-sided test of $H_0: p = 0.5$ against $H_a: p > 0.5$ at $\alpha = 0.05$ , using a random sample of 200 customers. The test rejects $H_0$ when $\hat{p}$ is 0.558 or more. The figure shows the sampling distribution of $\hat{p}$ if $H_0$ is true, and if in fact $p = 0.60$ .

5. What is the probability that the test makes a Type I error?

0.05, the significance level
0.11, the area to the left of the cutoff under the curve for $p = 0.60$
0.89, the area to the right of the cutoff under the curve for $p = 0.60$
0.558, the cutoff for rejecting $H_0$

Show the answer

A Type I error can happen only when $H_0$ is true, and then the test rejects with probability $\alpha$ . Here $\alpha = 0.05$ : the dark area to the right of the cutoff under the null curve.

Correct: 0.05, the significance level: Right: $P(\text{Type I error}) = \alpha$ .
0.11, the area to the left of the cutoff under the curve for $p = 0.60$ : That area is $\beta$ , the probability of a Type II error when $p = 0.60$ .
0.89, the area to the right of the cutoff under the curve for $p = 0.60$ : That is the power of the test against $p = 0.60$ .
0.558, the cutoff for rejecting $H_0$ : The cutoff is a value of $\hat{p}$ , not a probability.

6. Suppose that in fact $p = 0.60$ . Find the power of the test: the probability that $\hat{p}$ is 0.558 or more. Give three decimal places.

Type a number. A table answer and a calculator answer are both accepted.

Show the answer

If $p = 0.60$ , $\hat{p}$ is approximately normal with mean 0.60 and standard deviation $\sqrt{\frac{0.60(0.40)}{200}}$ $= 0.0346$ . Power $= P(\hat{p} \ge 0.558)$ $= P\left(Z \ge \frac{0.558 - 0.60}{0.0346}\right)$ $= P(Z \ge -1.21)$ $= 0.887$ .

Answer: 0.8873

7. The power against $p = 0.60$ is about 0.89. Which interpretation is correct?

If 60% prefer it, the test has an 89% chance of finding convincing evidence that over half do
There is an 89% chance that more than half of the company's customers prefer the new recipe
About 89% of customers will prefer the new recipe
If half of customers prefer it, the test has an 89% chance of a Type I error

Show the answer

Power is the probability of correctly rejecting $H_0$ when a particular alternative is true. If 60% of customers really prefer the new recipe, the test has about an 89% chance of finding convincing evidence that more than half do.

Correct: If 60% prefer it, the test has an 89% chance of finding convincing evidence that over half do: Right: power is the chance of detecting this alternative.
There is an 89% chance that more than half of the company's customers prefer the new recipe: Power is computed assuming a specific truth, $p = 0.60$ ; it is not a probability about $p$ .
About 89% of customers will prefer the new recipe: The 0.89 is a probability about the test's decision, not a share of customers.
If half of customers prefer it, the test has an 89% chance of a Type I error: If $p = 0.5$ , the chance of a Type I error is $\alpha = 0.05$ .

Part 9 · Summary

Summary

A Type I error rejects a true

H_0

(probability

\alpha

); a Type II error fails to reject a false

H_0

(probability

\beta

, against a specific alternative). Describe each with the condition first and its consequence in context. Power

= 1 - \beta

is the chance of detecting a real effect; it grows with the sample size, with

\alpha

, and with the distance between the true value and

p_0

Part 10 · Up next

What comes next

Part 11 · Connections

Connections

Question set for this topic