p-Values — AP® Statistics Lesson

Part 1 · Hook

Why this matters

Head office says 40% of customers buy a pastry with their coffee. At one store, only 105 of 300 sampled orders include one: 35%. Is that store really different, or would a gap that size show up by luck at a store that matches head office? The p-value puts a number on "by luck".

Part 2 · Before you start

What this builds on

Part 3 · Prerequisite check

Quick check before you start

1. A study tests $H_0: p = 0.40$ . Which large counts check is right for $n = 300$ ?

$300(0.40) = 120$ and $300(0.60) = 180$ , both at least 10
$300 \ge 30$
$300(0.35) = 105 \ge 10$ only
300 is less than 10% of the population

Show the answer

A test checks large counts with the null value $p_0 = 0.40$ , because it assumes $H_0$ is true.

Correct: $300(0.40) = 120$ and $300(0.60) = 180$ , both at least 10:
$300 \ge 30$ :
$300(0.35) = 105 \ge 10$ only:
300 is less than 10% of the population:

2. For a normal distribution with mean 0.40 and standard deviation 0.0283, about what is the z-score of 0.35?

$-1.77$
$1.77$
$-0.05$
$-0.35$

Show the answer

$z$ $= \frac{0.35 - 0.40}{0.0283}$ $= -1.77$ : value minus mean, over the standard deviation.

Correct: $-1.77$ :
$1.77$ :
$-0.05$ :
$-0.35$ :

Part 4 · See it

See it first

Two standard normal curves. Left, labeled one-sided, alternative p < 0.40: the area to the left of the test statistic z = −1.77 is shaded; it is the p-value, about 0.039. Right, labeled two-sided, alternative p is not 0.40: the areas to the left of z = −1.77 and to the right of z = 1.77 are both shaded; together they are the p-value, about 0.077, twice the one-sided value. — A p-value is a tail area of the null distribution: one tail for a one-sided alternative, both tails for a two-sided one. LevlPrep original diagram.

Part 5 · Step by step

How it works, step by step

Assume $H_0$ is true, so $p = p_0$ $\hat{p}$ is approximately normal with mean $p_0$ and standard deviation $\sqrt{p_0(1 - p_0)/n}$
Standardize the observed $\hat{p}$ the test statistic $z$ $= \frac{\hat{p} - p_0}{\sqrt{p_0(1 - p_0)/n}}$ says how many standard deviations it is from $p_0$
Find the area beyond $z$ in the direction of $H_a$ (both tails if two-sided)that area is the p-value: the chance of a result at least this extreme if $H_0$ is true
The p-value is smallthe result would be rare under $H_0$ , which is evidence against $H_0$ and for $H_a$ (but never proof)

Part 6 · Key ideas

Key ideas

Test statistic: $z$ $= \frac{\hat{p} - p_0}{\sqrt{p_0(1 - p_0)/n}}$ , with $p_0$ in the square root.
p-value: the probability, assuming $H_0$ is true, of a statistic at least as extreme as observed, in the direction of $H_a$ . Two-sided: double one tail.
Interpret: "Assuming the true proportion of all __ is $p_0$ , there is a __ probability of getting a sample proportion of __ or less (more) by chance in a random sample of __."
A simulation estimates a p-value: the share of simulated results, made assuming $H_0$ , at least as extreme as the real one.
Not the probability that $H_0$ is true. A large p-value does not prove $H_0$ .

Part 7 · Misconception

A common mistake

The wrong idea: A p-value of 0.0385 means there is a 3.85% chance that the null hypothesis is true.

What actually happens: The p-value is computed assuming $H_0$ is true. It is the probability of data at least this extreme if $H_0$ holds, not the probability that $H_0$ holds.

Part 8 · Check yourself

Check yourself

Exam-style questions. Anything you miss goes into your review queue.

Table

Pastries with coffee

A coffee chain's head office says that 40% of its customers order a pastry. A store manager suspects the proportion at her store is lower. She selects a random sample of 300 orders from the store's many orders this month, and 105 of them include a pastry. She tests $H_0: p = 0.40$ against $H_a: p < 0.40$ , where $p$ is the true proportion of all the store's orders this month that include a pastry. The conditions are met.

Sample results (invented data)
Orders sampled	With a pastry	Sample proportion
300	105	$\hat{p} = 0.35$

1. Calculate the standardized test statistic. Give two decimal places.

Type a number. A table answer and a calculator answer are both accepted.

Show the answer

$z$ $= \frac{\hat{p} - p_0}{\sqrt{p_0(1 - p_0)/n}}$ $= \frac{0.35 - 0.40}{\sqrt{0.40(0.60)/300}}$ $= \frac{-0.05}{0.0283}$ $= -1.77$ . The standard deviation uses $p_0$ , because the test assumes $H_0$ is true.

Answer: -1.77

2. Type the expression for the test statistic with the values substituted, as you would show it on the exam.

Type the expression as on the exam, with the values substituted.

Show the answer

(statistic − null value) / (standard deviation of the statistic): $\frac{0.35 - 0.40}{\sqrt{0.40(0.60)/300}}$ .

Answer: $\frac{0.35 - 0.4}{\sqrt{\frac{0.4 \cdot 0.6}{300}}}$

3. Find the p-value of the test. Give four decimal places.

Type a number. A table answer and a calculator answer are both accepted.

Show the answer

$H_a$ is $p < 0.40$ , so the p-value is the area to the left of $z = -1.77$ : $P(Z \le -1.77) = 0.0385$ . Technology (a 1-proportion z test) gives 0.0385. Table A with $z = -1.77$ gives 0.0384.

Answer: 0.0385

4. Build the interpretation of the p-value, 0.0385, in context.

Write the conclusion in context.

Show the answer

A p-value is a probability computed assuming $H_0$ is true: the chance of a sample result at least as extreme as the one observed, in the direction of $H_a$ .

A full-credit answer: Assuming the true proportion of all the store's orders that include a pastry is 0.40, there is a 0.0385 probability of getting a sample proportion of 0.35 or less by chance in a random sample of 300 orders.

5. What does the small p-value, 0.0385, suggest?

This result is unusual if $p = 0.40$ , which is evidence that $p < 0.40$
There is a 3.85% chance that the head office's figure of 40% is correct
About 3.85% of the store's orders include a pastry
The manager has proven that fewer than 40% of orders include a pastry

Show the answer

If the true proportion were 0.40, a sample proportion as low as 0.35 would happen only about 4% of the time. That makes 0.40 hard to believe, which is evidence for $H_a: p < 0.40$ .

Correct: This result is unusual if $p = 0.40$ , which is evidence that $p < 0.40$ : Right: a small p-value counts against $H_0$ and for $H_a$ .
There is a 3.85% chance that the head office's figure of 40% is correct: The p-value assumes the figure is correct; it is not the probability that it is.
About 3.85% of the store's orders include a pastry: The p-value is a probability about samples, not a proportion of orders.
The manager has proven that fewer than 40% of orders include a pastry: A small p-value is evidence, never proof; a sample this low can still happen when $p = 0.40$ .

6. For the banner test ( $z = 1.79$ , $H_a: p \ne 0.20$ ), find the p-value. Give four decimal places.

Type a number. A table answer and a calculator answer are both accepted.

Show the answer

Two-sided: count both tails. $P(Z \ge 1.79) = 0.0368$ , so the p-value is $2(0.0368) = 0.0736$ .

Answer: 0.0736

7. A student writes: "The p-value is 0.02, so there is a 2% chance that the null hypothesis is true." What is the error?

There is no error
It should say 98%, the complement, is the chance that $H_a$ is true
The p-value is computed assuming $H_0$ is true, so it is not the chance $H_0$ is true
A p-value has to be compared with 0.5 before it means anything

Show the answer

The p-value is computed by assuming $H_0$ is true. It is the probability of data this extreme or more under $H_0$ , not the probability that $H_0$ holds.

There is no error: This is the most common misreading of a p-value.
It should say 98%, the complement, is the chance that $H_a$ is true: The complement is not the probability that $H_a$ is true either.
Correct: The p-value is computed assuming $H_0$ is true, so it is not the chance $H_0$ is true: Right: the condition is "if $H_0$ is true", not "that $H_0$ is true".
A p-value has to be compared with 0.5 before it means anything: A p-value is compared with $\alpha$ , chosen in advance, not with 0.5.

Part 9 · Summary

Summary

A p-value measures how surprising the data are if

H_0

is true. Assuming

H_0

is true, standardize the sample proportion with

z

= \frac{\hat{p} - p_0}{\sqrt{p_0(1 - p_0)/n}}

and find the tail area in the direction of

H_a

(both tails for

\ne

). That area is the p-value: the chance of a result at least as extreme as the one observed if

H_0

is true. Small p-values are evidence against

H_0

; they are never the probability that

H_0

is true.

Part 10 · Up next

What comes next

Part 11 · Connections