Sampling and study design

What this area is about

Before any calculation, the way data were collected sets the limits of what they can say. A sample speaks for a population only if it was chosen fairly, and a difference between groups shows cause and effect only if the treatments were imposed and assigned at random. This area is about planning a study, and reading someone else’s, with those limits in view.

How these fit together

Definitions. Populations, samples and census separates the whole group from the part you measure. Parameters and statistics separates the number you want about the whole, such as \mu or p, from the one you can compute, such as \bar{x} or \hat{p}.

Collecting a sample. Random sampling lets chance choose the sample, so no one’s preferences tilt it. Bias names the ways a collection method leans the same way every time, which a bigger sample does not cure.

Studying cause. Observational studies and experiments says why a study shows cause only when it imposes treatments and assigns them at random, and experimental design builds that comparison from control, randomization, replication and blocking.

A path through it

Populations and samples come first. Parameters and statistics, and random sampling, follow in either order. Bias comes after random sampling, then observational studies and experiments, then experimental design.

Next: sampling distributions, confounding variables and null and alternative hypotheses.