Unit 1 Beta

Exploring One-Variable Data and Collecting Data: the one-page sheet

1.1 Introducing Statistics: What Can We Learn from Data?

Statistics answers questions about a population, the whole group of interest, using data from a sample, the part you actually measure. Answers vary, so a sample result is an estimate. It is useful only when the sample represents the population, and every result is stated in context.

Statistics is the science of collecting, summarizing and reading data to answer questions whose answers vary.
The population is the whole group a question is about. The sample is the part of it that data come from.
A study uses data from a sample to answer a question about a population. It works only if the sample represents the population.
State every result in context: who was studied, what was measured, and what the number tells you.
Another sample gives a slightly different result, so a sample result is an estimate for the population, not its exact value.

asking everyone would take too long and cost too much the answers vary from person to person, so you summarize them: 78 of 120 is 65% it is a good estimate for the population only if the sample represents the population so a conclusion about the population is an estimate, stated in context, never a certainty

statistics (field): The science of collecting, summarizing and interpreting data to answer questions whose answers vary from one individual to another.
data: The values recorded for the individuals in a study: answers, counts or measurements.
population: The whole group of individuals (people, animals or things) that a question is about.
sample: The part of a population from which data are actually collected.
in context: The real situation behind the numbers: who or what was studied and what was measured. Results are always stated in context.

1.2 Variables

Observational units are the individuals in a data set, and variables are the characteristics recorded for each. Categorical variables sort units into groups; quantitative ones record amounts, discrete or continuous. A parameter describes a population, a statistic describes a sample, and a distribution tells what values a variable takes and how often.

An observational unit is one individual the data describe: a person, a dog, a fish, a day. A variable is a characteristic recorded for each unit.
A categorical variable places each unit in a group (size group, yes or no). A quantitative variable records an amount, where arithmetic makes sense.
Quantitative variables are discrete (separate values, usually counts) or continuous (any value in an interval, usually measurements).
Numbers that are labels, such as ZIP codes or jersey numbers, are categorical.
A parameter describes a population; a statistic describes a sample. The distribution of a variable tells what values it takes and how often.

the individuals are the observational units each column is a variable, and its values can differ from row to row categorical and quantitative variables need different summaries and graphs it is a parameter (usually unknown) or a statistic (used to estimate the parameter)

observational unit: One individual that data describe, such as a person, an animal, an object or a day. Each row of a data table is one observational unit.
variable: A characteristic recorded for each observational unit, whose value can differ from one unit to another.
categorical variable: A variable whose values place each unit in a group or category, such as eye color or yes and no. Numbers used as labels, like ZIP codes, are categorical.
quantitative variable: A variable whose values are numbers that measure or count an amount, so arithmetic with them makes sense.
discrete variable: A quantitative variable that takes separate values with gaps between them, usually a count.
continuous variable: A quantitative variable that can take any value in an interval, usually a measurement such as time, length or weight.
parameter: A number that describes a population. It is usually unknown, because finding it needs data on every individual.
statistic (number): A number that describes a sample. It changes from sample to sample and is used to estimate a parameter.
distribution: The values a variable takes and how often it takes each value or range of values.

1.3 Tabular Representation and Summary Statistics for One Categorical Variable

A frequency table counts each category; a relative frequency divides each count by the total, so relative frequencies are between 0 and 1, add to 1, and compare groups of different sizes. In a sample, the relative frequency of a category is the sample proportion p-hat , a statistic that estimates the population proportion p .

A frequency table lists each category of a categorical variable and its count (frequency).
A relative frequency is a count divided by the total. Relative frequencies are between 0 and 1 and add to 1 when categories do not overlap.
Count = relative frequency × total, so a relative frequency table needs the total to get back to counts.
The sample proportion $\hat{p}$ is the relative frequency of a category in a sample, a statistic. The population proportion $p$ is the same quantity for the whole population, a parameter.

counting each category gives a frequency table the relative frequencies are parts of the whole, between 0 and 1, and they add to 1 samples of different sizes can be compared fairly it is a statistic that estimates p, the proportion in the whole population

frequency table: A table that lists each category of a categorical variable with its frequency, the number of individuals in that category.
relative frequency: A count divided by the total number of individuals: the part of the whole in one category, written as a decimal, fraction or percent.
sample proportion: The part of a sample that falls in a category, <math class="stat-m"><semantics><mrow><mover accent="true"><mi>p</mi><mo stretchy="false">^</mo></mover></mrow><annotation encoding="application/x-tex">\hat{p}</annotation></semantics></math><wbr><math class="stat-m"><semantics><mrow><mo form="infix">=</mo><mfrac><mtext>count in the category</mtext><mi>n</mi></mfrac></mrow><annotation encoding="application/x-tex">= \frac{\text{count in the category}}{n}</annotation></semantics></math>, read "p-hat". It is a statistic.
population proportion: The part of a whole population that falls in a category, written <math class="stat-m"><semantics><mrow><mi>p</mi></mrow><annotation encoding="application/x-tex">p</annotation></semantics></math>. It is a parameter, usually unknown and estimated by <math class="stat-m"><semantics><mrow><mover accent="true"><mi>p</mi><mo stretchy="false">^</mo></mover></mrow><annotation encoding="application/x-tex">\hat{p}</annotation></semantics></math>.

1.4 Graphical Representations for One Categorical Variable

Bar charts and pie charts display one categorical variable. A bar’s height is its count or relative frequency measured from 0; a pie’s slices are parts of one whole, so every individual must be in exactly one category. Compare groups of different sizes with relative frequencies, and check axes before trusting a picture.

A bar chart shows one bar per category, with height equal to the count or relative frequency. The vertical axis starts at 0.
A pie chart shows each category as a slice of one circle, sized by its relative frequency. It needs categories that do not overlap and cover everyone.
Bar charts make it easier to compare categories; pie charts show how parts make up a whole.
To compare groups of different sizes, graph relative frequencies, and describe what the graph shows with numbers, in context.

the bar’s height is the category’s count or relative frequency, measured from 0 bars can go in any order and are drawn with gaps between them the parts make one whole, so a pie chart can show them as slices the picture misleads, so start at 0 and graph relative frequencies to compare groups

bar chart: A graph of a categorical variable with one bar per category, separated by gaps, whose height (from 0) is the category’s count or relative frequency.
pie chart: A circle cut into slices, one per category, each slice sized by the category’s relative frequency. It needs categories that together make up one whole.

1.5 Graphical Representations for One Quantitative Variable

Dotplots and stemplots show every value of a small quantitative data set; histograms group values into equal bins and show the count in each, and their look depends on the bin width. A cumulative relative frequency graph shows the proportion of values at or below each point, so a proportion between two values is a difference of heights.

A dotplot shows each value as a dot above a number line; repeated values stack. Best for small data sets.
A stemplot splits each value into a stem (leading digits) and a leaf (last digit). It keeps every value; always give a key.
A histogram groups values into equal bins; each bar’s height is the bin’s count or relative frequency, and the bars touch. Its look depends on the bin width.
A cumulative relative frequency graph gives, at each value, the proportion of the data at or below that value. The proportion between two values is the difference of the heights.

a dotplot puts one dot per value above its place on the line a histogram groups values into equal bins and draws a bar for each bin’s count the same data can look bumpy with narrow bins or smooth with wide ones a cumulative relative frequency graph shows the proportion of values at or below each point

dotplot: A graph of a quantitative variable that puts one dot above a number line for each value; repeated values stack up.
stemplot: A display of a quantitative variable that splits each value into a stem (its leading digits) and a leaf (its last digit), so every value is kept. It needs a key, such as 7 | 4 stands for 74.
histogram: A graph of a quantitative variable that groups values into equal-width bins on a number line, with touching bars whose heights are the counts or relative frequencies in each bin.
cumulative relative frequency graph: A graph that shows, for each value, the proportion of the data at or below that value (the relative frequencies added up from the left).

1.6 Descriptions for One Quantitative Variable Distributions

To describe a quantitative distribution, give its shape (symmetric, skewed toward the tail, unimodal, bimodal or uniform), its center and spread with numbers, and any unusual features such as outliers, gaps and clusters. Every description names the variable and its units, so it is in context.

Describe a quantitative distribution with four features: shape, center, spread and unusual features, all in context with numbers.
Shape: symmetric, skewed right or left (named for the long tail), unimodal or bimodal, or uniform (bars about level).
Spread (variability) is how far apart the values are; give it with numbers, such as "most values from 5 to 30 minutes".
Unusual features: an outlier falls far from the overall pattern; a gap is an empty stretch; a cluster is a separate bunch of values.

you can read its shape: symmetric, skewed, one peak or two, or level the distribution is skewed toward the tail: right for large values, left for small values you describe the center and the spread with numbers you name them as gaps, clusters or possible outliers, and every feature is stated in context

shape: The overall form of a quantitative distribution on a graph: whether it is symmetric or skewed, how many peaks it has, or whether it is level (uniform).
symmetric: A distribution whose left and right sides are roughly mirror images of each other around its center.
skewed: A distribution with a long tail on one side. It is skewed right when the tail stretches toward larger values, skewed left when the tail stretches toward smaller values.
unimodal: The number of peaks (modes) in a distribution: unimodal has one, bimodal has two, multimodal has more.
uniform distribution: A distribution in which all values occur about equally often, so its histogram has bars of about the same height.
center: A typical or middle value of a distribution, around which the data are located.
variability: Variability: how far apart the values of a distribution are from each other, or how much they vary.
outlier: A value that falls far from the overall pattern of a distribution. It should be investigated, not simply removed.
gap: A gap is a stretch of the number line with no values; a cluster is a bunch of values set apart from the rest by gaps.

1.7 Summary Statistics for One Quantitative Variable

The mean and standard deviation use every value, so outliers and skew pull them; the median and IQR are resistant. Quartiles are the medians of the two halves, and the 1.5×IQR fences or the 2 standard deviation rule flag outliers. Adding a constant shifts center but not spread; multiplying by a constant scales both.

Mean $\bar{x} = \frac{\sum x_i}{n}$ ; median: the middle of the ordered values (the average of the two middle ones when n is even).
Quartiles: $Q_1$ and $Q_3$ are the medians of the lower and upper halves (leave out the median when n is odd). $\text{IQR} = Q_3 - Q_1$ ; range = max − min.
Standard deviation $s$ $= \sqrt{\frac{\sum (x_i - \bar{x})^2}{n - 1}}$ , the typical distance from the mean; variance is $s^2$ . Population versions: $\mu$ and $\sigma$ .
Outliers: beyond $Q_1 - 1.5 \times \text{IQR}$ or $Q_3 + 1.5 \times \text{IQR}$ , or more than 2 standard deviations from the mean.
Adding a constant shifts the center but not the spread; multiplying by a constant multiplies both.

one extreme value pulls it toward the tail they barely move when an extreme value changes: they are resistant it measures typical distance from the mean, and an outlier inflates it report the median and IQR; for roughly symmetric data, the mean and standard deviation

mean: The sum of the values divided by how many there are: <math class="stat-m"><semantics><mrow><mover accent="true"><mi>x</mi><mo stretchy="true">¯</mo></mover><mo>=</mo><mfrac><mrow><mo largeop="true" movablelimits="false">∑</mo><msub><mi>x</mi><mi>i</mi></msub></mrow><mi>n</mi></mfrac></mrow><annotation encoding="application/x-tex">\bar{x} = \frac{\sum x_i}{n}</annotation></semantics></math> for a sample. It is not resistant.
population mean: The mean of a whole population, written <math class="stat-m"><semantics><mrow><mi>μ</mi></mrow><annotation encoding="application/x-tex">\mu</annotation></semantics></math> (mu). It is a parameter, estimated by the sample mean <math class="stat-m"><semantics><mrow><mover accent="true"><mi>x</mi><mo stretchy="true">¯</mo></mover></mrow><annotation encoding="application/x-tex">\bar{x}</annotation></semantics></math>.
median: The middle value of the ordered data, or the average of the two middle values when n is even. Half the data are at or below it. It is resistant.
quartile: The first quartile <math class="stat-m"><semantics><mrow><msub><mi>Q</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">Q_1</annotation></semantics></math> is the median of the lower half of the ordered data and the third quartile <math class="stat-m"><semantics><mrow><msub><mi>Q</mi><mn>3</mn></msub></mrow><annotation encoding="application/x-tex">Q_3</annotation></semantics></math> is the median of the upper half (the median itself is left out of both halves when n is odd).
percentile: The pth percentile is the value with p percent of the data at or below it.
range: The maximum minus the minimum: a single number that measures spread. It is not resistant.
interquartile range: The interquartile range, <math class="stat-m"><semantics><mrow><mtext>IQR</mtext><mo>=</mo><msub><mi>Q</mi><mn>3</mn></msub><mo>−</mo><msub><mi>Q</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">\text{IQR} = Q_3 - Q_1</annotation></semantics></math>: the spread of the middle half of the data. It is resistant.
standard deviation: A measure of the typical distance of the values from the mean: <math class="stat-m"><semantics><mrow><mi>s</mi></mrow><annotation encoding="application/x-tex">s</annotation></semantics></math><wbr><math class="stat-m"><semantics><mrow><mo form="infix">=</mo><msqrt><mfrac><mrow><mo largeop="true" movablelimits="false">∑</mo><mo stretchy="false">(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>−</mo><mover accent="true"><mi>x</mi><mo stretchy="true">¯</mo></mover><msup><mo stretchy="false">)</mo><mn>2</mn></msup></mrow><mrow><mi>n</mi><mo>−</mo><mn>1</mn></mrow></mfrac></msqrt></mrow><annotation encoding="application/x-tex">= \sqrt{\frac{\sum (x_i - \bar{x})^2}{n - 1}}</annotation></semantics></math> for a sample. It is 0 only when every value is the same, and it is not resistant.
variance: The square of the standard deviation, <math class="stat-m"><semantics><mrow><msup><mi>s</mi><mn>2</mn></msup></mrow><annotation encoding="application/x-tex">s^2</annotation></semantics></math><wbr><math class="stat-m"><semantics><mrow><mo form="infix">=</mo><mfrac><mrow><mo largeop="true" movablelimits="false">∑</mo><mo stretchy="false">(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>−</mo><mover accent="true"><mi>x</mi><mo stretchy="true">¯</mo></mover><msup><mo stretchy="false">)</mo><mn>2</mn></msup></mrow><mrow><mi>n</mi><mo>−</mo><mn>1</mn></mrow></mfrac></mrow><annotation encoding="application/x-tex">= \frac{\sum (x_i - \bar{x})^2}{n - 1}</annotation></semantics></math> for a sample.
population standard deviation: The standard deviation of a whole population, written <math class="stat-m"><semantics><mrow><mi>σ</mi></mrow><annotation encoding="application/x-tex">\sigma</annotation></semantics></math> (sigma). It is a parameter, estimated by the sample standard deviation <math class="stat-m"><semantics><mrow><mi>s</mi></mrow><annotation encoding="application/x-tex">s</annotation></semantics></math>.
1.5 IQR rule: A value is an outlier if it is below <math class="stat-m"><semantics><mrow><msub><mi>Q</mi><mn>1</mn></msub><mo>−</mo><mn>1.5</mn><mo>×</mo><mtext>IQR</mtext></mrow><annotation encoding="application/x-tex">Q_1 - 1.5 \times \text{IQR}</annotation></semantics></math> or above <math class="stat-m"><semantics><mrow><msub><mi>Q</mi><mn>3</mn></msub><mo>+</mo><mn>1.5</mn><mo>×</mo><mtext>IQR</mtext></mrow><annotation encoding="application/x-tex">Q_3 + 1.5 \times \text{IQR}</annotation></semantics></math> (the fences). Another rule flags values more than 2 standard deviations from the mean.
resistant: A summary is resistant if extreme values barely change it. The median and IQR are resistant; the mean, standard deviation and range are not.

1.8 Graphical Representations of Summary Statistics for One Quantitative Variable

A boxplot pictures the five-number summary: a box from Q_1 to Q_3 split at the median, whiskers to the most extreme values inside the 1.5×IQR fences, and outliers marked separately. It shows center, the spread of the middle half and skew, and about a quarter of the data lies in each section, but it hides peaks and gaps.

The five-number summary is minimum, $Q_1$ , median, $Q_3$ , maximum. Each quarter of the data lies between two neighbors.
A boxplot draws a box from $Q_1$ to $Q_3$ with a line at the median, whiskers to the most extreme values inside the fences, and outliers as separate points.
Skew shows as a longer whisker and a median closer to one end of the box: skewed right when the right side is stretched.
A boxplot cannot show modality, gaps or clusters, so never describe a boxplot as having one or two peaks.

these five numbers split the ordered data into four quarters the box holds the middle half of the data, and its length is the IQR values beyond them are marked as outliers, and each whisker stops at the most extreme value inside its fence it shows center, spread and skew, but it cannot show peaks, gaps or the mean

five-number summary: The minimum, first quartile <math class="stat-m"><semantics><mrow><msub><mi>Q</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">Q_1</annotation></semantics></math>, median, third quartile <math class="stat-m"><semantics><mrow><msub><mi>Q</mi><mn>3</mn></msub></mrow><annotation encoding="application/x-tex">Q_3</annotation></semantics></math> and maximum of a data set. Together they split the ordered data into four quarters.
boxplot: A graph of the five-number summary: a box from <math class="stat-m"><semantics><mrow><msub><mi>Q</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">Q_1</annotation></semantics></math> to <math class="stat-m"><semantics><mrow><msub><mi>Q</mi><mn>3</mn></msub></mrow><annotation encoding="application/x-tex">Q_3</annotation></semantics></math> with a line at the median, whiskers to the most extreme values inside the <math class="stat-m"><semantics><mrow><mn>1.5</mn><mo>×</mo><mtext>IQR</mtext></mrow><annotation encoding="application/x-tex">1.5 \times \text{IQR}</annotation></semantics></math> fences, and outliers drawn as separate points.

1.9 Comparisons of the Distributions for One Quantitative Variable

To compare distributions, describe each feature (shape, center, spread, outliers) with a comparing word and numbers, in context. To compare single values from different distributions, use relative position: the z-score z = (x − x-bar)/s counts standard deviations from the mean, and a percentile gives the percent of values at or below.

Compare distributions on every feature: shape, center, spread and outliers, each with a comparing word and numbers, in context.
A z-score is $z = \frac{x - \bar{x}}{s}$ (or $\frac{x - \mu}{\sigma}$ for a population): how many standard deviations a value is above (+) or below (−) the mean. It has no units.
Relative position compares values from different groups: z-scores and percentiles do this; raw values do not.
The median of combined groups is not the average of their medians.

graph them on one scale, such as parallel boxplots or stacked dotplots use comparing words (greater, less variable) with numbers for each feature, in context compare their positions, not their raw sizes the z-score puts both on one scale: standard deviations above or below their own mean

z-score: The number of standard deviations a value is above or below the mean: <math class="stat-m"><semantics><mrow><mi>z</mi><mo>=</mo><mfrac><mrow><mi>x</mi><mo>−</mo><mover accent="true"><mi>x</mi><mo stretchy="true">¯</mo></mover></mrow><mi>s</mi></mfrac></mrow><annotation encoding="application/x-tex">z = \frac{x - \bar{x}}{s}</annotation></semantics></math> (<math class="stat-m"><semantics><mrow><mfrac><mrow><mi>x</mi><mo>−</mo><mi>μ</mi></mrow><mi>σ</mi></mfrac></mrow><annotation encoding="application/x-tex">\frac{x - \mu}{\sigma}</annotation></semantics></math> for a population). Positive is above the mean, negative below; it has no units.
relative position: Where a value stands within its own distribution, measured by a percentile or a z-score, so values from different distributions can be compared.

1.10 The Investigative Question Revisited and Data Collection

An investigative question names the population, the variable, the parameter and the kind of conclusion wanted. A census measures everyone; sample surveys and observational studies record variables without imposing anything, so they show relationships but confounding blocks cause. Only an experiment, with conditions decided by chance, supports a cause-and-effect conclusion.

An investigative question names the population, the variable, the parameter (such as a proportion or a typical value) and whether the goal is to describe or to look for a cause.
A census collects data from every member of the population; a sample survey asks questions of a sample.
An observational study records variables without imposing anything (looking back: retrospective; following forward: prospective). An experiment imposes conditions and measures the response.
The explanatory variable may help explain changes in the response variable. A confounding variable is related to both, so its effect cannot be separated.
A cause-and-effect conclusion needs an experiment in which chance decides who gets which condition.

it tells you what to measure, on whom, and which parameter you are after a census answers the question directly you can describe the population and show how variables are related confounding blocks a cause-and-effect conclusion; that needs an experiment with conditions decided by chance

investigative question: A question that data can answer: it names the population, the variable, what to estimate (such as a proportion or a typical value) and whether the goal is to describe or to find a cause.
census: A study that collects data from every member of the population.
observational study: A study that records variables on individuals without imposing any condition on them. A retrospective study looks back at existing records; a prospective study follows individuals forward in time.
experiment: A study in which the researcher deliberately imposes conditions on individuals and measures their responses.
sample survey: A sample survey collects data by asking questions of a sample of individuals. It is an observational study.
explanatory variable: The explanatory variable may help explain or predict changes in the response variable, the outcome being measured.
confounding variable: A confounding variable is related to both the explanatory and the response variable, so its effect on the response cannot be separated from the explanatory variable’s effect.
cause-and-effect conclusion: A cause-and-effect conclusion says that changing the explanatory variable changes the response. It needs an experiment in which chance decides who gets which condition; observational studies cannot rule out confounding.

1.11 Random Sampling

Random sampling uses chance to choose individuals from a sampling frame. A simple random sample gives every group of n the same chance; a stratified sample takes an SRS within each stratum of similar individuals; a cluster sample takes whole groups chosen at random; a systematic sample takes every kth individual after a random start. Write procedures as label, generate, skip repeats, select.

A simple random sample (SRS) of size n gives every group of n individuals the same chance of being chosen. Procedure: label, generate random numbers, skip repeats, select.
Sampling without replacement means each individual can be chosen once; with replacement allows repeats.
A stratified random sample splits the population into strata of similar individuals and takes an SRS from each.
A cluster sample chooses whole groups at random and measures everyone in them. A systematic random sample takes every kth individual after a random start.
The sampling frame is the list the sample is drawn from.

no one’s preferences decide who is in it, so it tends to represent the population you get a simple random sample: every group of n has the same chance every group is represented, and estimates vary less when strata differ sampling is cheaper and easier, at some cost in precision or with a risk from patterns in the list

random: Choosing individuals by a chance process, such as a random number generator, so that no one’s judgment decides who is chosen.
simple random sample: A simple random sample (SRS) of size n is chosen so that every group of n individuals in the population has the same chance of being the sample.
without replacement: Sampling without replacement means an individual can be chosen once; with replacement, an individual is returned and can be chosen again.
stratified random sample: A stratified random sample splits the population into strata, groups of similar individuals, and takes a simple random sample from each stratum.
cluster sample: A cluster sample splits the population into groups (clusters), chooses some clusters at random, and measures every individual in the chosen clusters.
systematic random sample: A systematic random sample chooses a random starting point in an ordered list and then takes every kth individual.
sampling frame: The list of individuals from which a sample is actually chosen, such as a roster, a map or a list of addresses.

1.12 Potential Problems with Sampling

Bias is a systematic error in a sampling method. Voluntary response and convenience samples are biased because they are not random; undercoverage, nonresponse and response bias can affect even random samples. To explain a bias, name it, say how the sample differs from the population and in which direction, and remember that a bigger sample does not cure it.

A sampling method is biased if it systematically overestimates or underestimates the value it is trying to estimate.
Voluntary response samples (people choose to answer) and convenience samples (easy to reach) are not random and are usually biased.
Undercoverage: part of the population is missing from the frame. Nonresponse: chosen individuals do not respond. Response bias: answers are not true, from leading wording, sensitive questions or pressure.
Name the bias, explain how the sample differs from the population, and give the likely direction. A larger sample never removes bias.

over-represents some kinds of people: voluntary response or convenience sampling those people have no chance of being chosen: undercoverage nonresponse and response bias, even in a random sample the method is biased, and a larger sample does not fix it

bias: A sampling method is biased if it systematically overestimates or underestimates the population value, in the same direction sample after sample.
voluntary response sample: A voluntary response sample is made of people who choose themselves, for example by answering an online poll. It overrepresents people with strong opinions.
convenience sample: A convenience sample is made of individuals who are easy to reach, such as shoppers at one store. It is not random and is usually biased.
undercoverage: Undercoverage happens when part of the population is left out of the sampling frame, so those individuals have no chance of being chosen.
nonresponse: Nonresponse happens when individuals chosen for a sample cannot be reached or do not respond. If they differ from those who respond, the results are biased.
response bias: Response bias happens when answers do not reflect the truth, because of leading question wording, sensitive questions, or pressure from the interviewer or the setting.

1.13 Experimental Design

An experiment imposes treatments on experimental units and measures a response. Good designs compare treatments (often with a control or placebo), assign them at random, replicate, control other variables and blind where possible. Completely randomized, randomized block and matched pairs designs differ in how units are grouped before randomizing. Random assignment allows cause and effect; random sampling allows generalizing.

Experimental units (or subjects) receive treatments, the conditions imposed; the response variable is measured afterward. A control group gets no treatment, a placebo or the standard one.
Principles: compare treatments, use random assignment, use replication (enough units per treatment), and control other variables. Blinding: single-blind or double-blind.
Completely randomized design: assign every unit by chance. Randomized block design: group similar units into blocks, randomize within each. Matched pairs: pairs of similar units, or each unit gets both treatments in random order.
A difference is statistically significant when it is too large to be explained by chance alone. Random assignment allows cause; random sampling allows generalizing.

the comparison shows the effect of the treatment, not of time or expectation other variables balance between groups on average, so they cannot explain a difference chance differences average out and expectations do not bias the response conclude the treatment caused it, for units like these; generalizing needs a random sample

experimental unit: The individual (person, animal, plant or object) to which a treatment is assigned in an experiment. Experimental units that are people are called subjects.
treatment: A specific condition imposed on experimental units. A factor is an explanatory variable in an experiment; its levels are the values it is set to.
control group: A group that receives no treatment, a placebo or the standard treatment, to compare with the treatment group. A placebo is a fake treatment; the placebo effect is a response to it caused by expectation.
random assignment: Using a chance process to decide which experimental units receive which treatment, so that other variables are balanced between groups on average.
replication: Using enough experimental units in each treatment group that chance differences between units tend to average out.
blinding: Keeping subjects (single-blind) or both subjects and the people who interact with them or measure the response (double-blind) from knowing who received which treatment.
completely randomized design: An experimental design in which every experimental unit is assigned to a treatment at random, with no grouping first.
randomized block design: A randomized block design groups experimental units that are similar in a way expected to affect the response into blocks, then assigns treatments at random within each block.
matched pairs design: A design in which units are put in pairs of similar units and chance decides which unit of each pair gets which treatment, or each unit receives both treatments in a random order.
statistically significant: A difference between treatment groups is statistically significant when it is too large to be explained by the chance variation of random assignment alone.