Practice this lesson

Unit 3 · Topic 3.6 Beta

p-Values

4 min read · freeNot practiced

Share to Google Classroom (opens in a new tab)

The setup of a test fixes the hypotheses and the standard of evidence. Now the data arrive, and we need one number that says how surprising they are if the null hypothesis is true. That number is the p-value. This topic is about computing it and, just as important, saying what it means.

The logic: assume H0H_0, then ask how surprising the data are

A significance test argues like this. Suppose H0H_0 is true. Then the sampling distribution of p^\hat{p} is centered at p0p_0, and we know its spread. If our sample proportion lands somewhere typical in that distribution, the data are consistent with H0H_0. If it lands far out in a tail, in the direction HaH_a points, then either something rare happened or H0H_0 is wrong. The rarer the result under H0H_0, the stronger the evidence against it.

The test statistic

To measure "how far out", standardize. The reference sheet gives the general form of a standardized test statistic:

test statistic=statistic−parameterstandard error of the statistic\text{test statistic} = \frac{\text{statistic} - \text{parameter}}{\text{standard error of the statistic}}

For one proportion, the statistic is p^\hat{p}, the parameter is its null value p0p_0, and because we are assuming H0H_0 is true, the spread is computed with p0p_0 too:

z=p^−p0p0(1−p0)nz = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1 - p_0)}{n}}}

Using p^\hat{p} inside the square root is the most common error on this calculation. An interval uses p^\hat{p} there because it has no null value; a test has one and must use it.

The p-value

The p-value is the probability, assuming H0H_0 is true, of getting a statistic at least as extreme as the one observed, in the direction of HaH_a. Which area you find depends on the alternative:

Alternativep-value
Ha:p<p0H_a: p < p_0area to the left of zz: P(Z≤z)P(Z \le z)
Ha:p>p0H_a: p > p_0area to the right of zz: P(Z≥z)P(Z \ge z)
Ha:p≠p0H_a: p \ne p_0both tails: 2P(Z≥|z|)2P(Z \ge |z|)
Two standard normal curves. Left, labeled one-sided, alternative p < 0.40: the area to the left of the test statistic z = −1.77 is shaded; it is the p-value, about 0.039. Right, labeled two-sided, alternative p is not 0.40: the areas to the left of z = −1.77 and to the right of z = 1.77 are both shaded; together they are the p-value, about 0.077, twice the one-sided value.
Figure 1. The same test statistic, z = −1.77, gives a one-sided p-value of 0.039 and a two-sided p-value of 0.077. LevlPrep original diagram.

Worked example. A coffee chain says 40% of its customers order a pastry. A store manager suspects her store is lower. In a random sample of 300 of the store's orders, 105 include a pastry. Test H0:p=0.40H_0: p = 0.40 against Ha:p<0.40H_a: p < 0.40, where pp is the true proportion of all the store's orders that include a pastry. (Random, 10% and large counts, 300(0.40)=120300(0.40) = 120 and 300(0.60)=180300(0.60) = 180, are met.)

Statistic. p^=105300=0.35\hat{p} = \frac{105}{300} = 0.35.

Test statistic. zz=0.35−0.400.40(0.60)/300= \frac{0.35 - 0.40}{\sqrt{0.40(0.60)/300}}=−0.050.0283= \frac{-0.05}{0.0283}=−1.77= -1.77.

p-value. HaH_a says "less than", so find the left tail: P(Z≤−1.77)=0.0385P(Z \le -1.77) = 0.0385 (technology: a 1-proportion z test gives z=−1.77z = -1.77, p-value 0.0385; Table A gives 0.0384).

Interpretation. Assuming the true proportion of all the store's orders that include a pastry is 0.40, there is a 0.0385 probability of getting a sample proportion of 0.35 or less by chance in a random sample of 300 orders.

The interpretation has four parts: the condition ("assuming the true proportion is 0.40"), the probability, the statistic and direction ("0.35 or less"), and the sample ("a random sample of 300 orders"). For a two-sided test, say "at least as far from 0.40 as the observed value, in either direction".

A p-value from a simulation

You can also estimate a p-value without a formula. Simulate many samples in a world where H0H_0 is true, then count the simulated results at least as extreme as the real one. A player who claims to make 75% of her free throws makes 31 of 50 (p^=0.62\hat{p} = 0.62). If 3 of 100 simulated sets of 50 shots by a 75% shooter come out at 0.62 or lower, the estimated p-value is 3100=0.03\frac{3}{100} = 0.03. The dotplot of simulated results is centered at 0.75, not at 0.62, because it is built assuming H0H_0.

What a p-value is not

  • It is not the probability that H0H_0 is true. It is computed by assuming H0H_0 is true.
  • A large p-value does not show that H0H_0 is true. It shows that the data are consistent with it.
  • A small p-value is evidence against H0H_0, not proof. Rare results do happen.

Smaller p-values come from results farther from p0p_0 in the direction of HaH_a, and from larger samples, which shrink the standard deviation and so stretch the same difference into a larger zz. In the next topic you will compare the p-value with α\alpha to reach a decision.

Spot a mistake on this page?