Simple random samples and the sampling frame
You can only ask a few members of a group. How do you choose them so that nobody’s preferences, not even your own, decide who is asked?
Where you are 4 in Random sampling
When you reach for it
You want to know something about a whole population and can only ask or measure a sample of it. Convenience and voluntary response samples let ease or eagerness decide who is in the sample, and the result leans toward whichever corner of the population was reached.
A simple random sample is the fix: chance, and nothing else, decides who is in it, so nobody’s preferences can tilt it. Reach for it when you have a list of the whole population, or can make one, and want a result that speaks for all of it by a method anyone can check was fair.
The procedure
Day 1: what makes a sample simple random, and how to draw one
The definition. A sample of n members from a population of N members is a simple random sample when a chance process chooses it so that
- every member has the same chance of being in the sample, and
- every group of n members has the same chance of being the sample.
No member can be chosen twice (the sample is drawn without replacement), so it always holds n different members.
The second condition is the heart of it. Choose 2 of four friends, Montague, Nell, Orlaith and Piers, by drawing two slips from a hat. These are all the groups that can come out:
| Group | Group | Group |
|---|---|---|
| Montague, Nell | Montague, Orlaith | Montague, Piers |
| Nell, Orlaith | Nell, Piers | Orlaith, Piers |
The hat favors no slip, so it favors no pair: each of the 6 groups has chance \displaystyle \frac{1}{6}. Each friend is in 3 of them, so each friend’s chance is \displaystyle \frac{3}{6} = \frac{1}{2}, which is \displaystyle \frac{2}{4}: 2 places shared among 4 friends. In general, every draw fills exactly n places and no member is favored, so the places are shared evenly, and each member’s chance is
The first condition does not give the second: Example 4 is a plan where every member has the same chance and yet most groups can never be the sample.
Why chance gives a fair picture. When a person chooses, his or her preferences leak in. When chance chooses, nothing about a member, neither height nor job nor opinion, makes that member more likely to be picked. The result in populations and samples makes this exact: averaged over all the possible simple random samples, the sample percent equals the population percent. Example 1 works it out for an orchestra. A single sample can still land some way off; how far is sampling variability. Random sampling tends to give a representative sample. It does not promise one.
Names in a hat. For a small population, write each name on a slip. Each detail protects the definition. One slip per member, so nobody has two chances or none. Slips the same size, folded alike, so none is easier to grab. Mix well, so the last slips put in are not lying on top. Draw without looking, so the drawer has no say. Keep each slip out once drawn, and stop at n.
Random digits. For a larger population, number the members and let chance pick numbers. With a table of random digits every label needs the same number of digits: thirty members are 01 to 30, read two digits at a time; two hundred trees are 001 to 200, read three at a time. Otherwise the digits 3 then 0 could mean member 3 or member 30. How a random digit table works and how to read along it is in designing a simulation. Here, read blocks as long as the labels. A block that is a member’s label picks that member; a block that is nobody’s label, too large or all zeros, is skipped; a block repeating a member already chosen is skipped; stop at n. Skipping is fair because every block from 00 to 99 is equally likely, so among the blocks kept each member’s label is as likely as any other. It costs digits, not fairness.
A calculator or computer. Many calculators and spreadsheets give a random whole number from 1 to N, each equally likely. Ignore any number already chosen and stop when n different numbers have come up. The labels need not have equal lengths here.
Day 2: the sampling frame, and what chance cannot fix
The frame. Nobody draws from a population directly, only from a list of it: a register, a membership list, a map. The list the sample is actually drawn from is the sampling frame. The ideal frame lists every member of the population exactly once and nothing else. Real frames fall short in three ways.
- It misses members. A member not on the list has chance 0, however fair the draw. If the missing members differ in the thing measured, the result leans, as a convenience sample’s does. This is undercoverage, treated fully in undercoverage and nonresponse.
- It holds entries that are not members, such as a person who has moved away. When one is drawn, set it aside and draw again at random from the frame. A convenient replacement would let a person decide who is in the sample.
- It lists some members twice. A member listed twice has two slips in the hat. Remove repeats before drawing.
The sample speaks for the frame. Chance makes the sample stand in for the list it was drawn from. If the list is not the population, a larger sample from the same list only stands in for the wrong group more firmly.
Chance fixes who is asked, not how they answer. A question worded to push an answer is response bias, and random sampling does nothing about it. Nor does it make the chosen members reply. Chosen members who never answer are nonresponse, and filling their places with volunteers turns the sample back into a voluntary response sample.
Worked examples
Day 1: five players for the new chairs
An orchestra has 30 members, and 12 of them play string instruments. A chair maker lends 5 new chairs, and 5 members are to sit in them for a two weeks and report back. The conductor offers to pick five sensible players. Explain why a hat is better, describe the draw, and find each member’s chance and the average number of string players in the sample.
SolutionComfort depends on the person: height, instrument, how a player sits to play. If the conductor chooses, her idea of “sensible” decides who reports, perhaps players who never complain, and the report describes them rather than the orchestra. A hat leaves her nothing to decide.
Put 30 name slips, alike in size and folding, in a hat, mix well, and draw 5 without looking, keeping each out. Every group of 5 is then equally likely, and each member’s chance is
Averaged over all the possible samples, the sample holds \displaystyle 12 \times \frac{1}{6} = 2 string players, so \displaystyle \frac{2}{5} = 40\% of it, matching \displaystyle \frac{12}{30} = 40\% of the orchestra. A single draw could still hold none or five.
Day 1: reading pairs of random digits
The orchestra’s 30 members are numbered 01 to 30 in alphabetical order. Use this row of random digits, from its first digit, to choose 5 members.
The labels have two digits, so read pairs straight across the gaps, which only help the eye: 93 takes its 9 from the first group and its 3 from the second.
| Pair | Decision | Reason |
|---|---|---|
| 62 | skip | no member 62 |
| 17 | pick | member 17 |
| 93 | skip | no member 93 |
| 05 | pick | member 05 |
| 30 | pick | member 30 |
| 41 | skip | no member 41 |
| 17 | skip | already chosen |
| 88 | skip | no member 88 |
| 26 | pick | member 26 |
| 00 | skip | no member 00 |
| 74 | skip | no member 74 |
| 09 | pick | member 09 |
The sample is members 05, 09, 17, 26 and 30; the last digit is not needed. Only 30 of the 100 possible pairs are labels, so most pairs are skipped. That is slow, not unfair.
Day 1: twenty bottles from a shift
A factory fills 500 bottles in a shift and prints a number on each, 1 to 500, in filling order. An inspector will measure 20 of them. Her calculator’s random whole numbers from 1 to 500 begin 214, 37, 488, 214, 151, 500. Which bottles are chosen so far, what is each bottle’s chance, and why not simply measure bottles 1 to 20?
SolutionThe second 214 repeats a chosen bottle and is ignored, so the first five are 214, 37, 488, 151 and 500; she continues until 20 different bottles are chosen. Each bottle’s chance is
Bottles 1 to 20 are the first twenty filled. If the machine drifts during the shift, say a nozzle slowly clogs, they show only how the shift began. A fixed set also breaks the definition: that one group would be the sample with certainty and every other group would have chance 0.
Day 1: two plans that give every tree the same chance
A park’s 200 trees stand in 20 columns with 10 trees in each column; one column runs beside a stream. A ranger will measure the trunks of 10 trees. Plan A: number the trees 1 to 200 and let a calculator pick 10 different numbers. Plan B: number the columns 1 to 20, let a calculator pick one, and measure all 10 trees in it. Find each tree’s chance under each plan, and decide which gives a simple random sample.
SolutionUnder Plan A each tree’s chance is
Under Plan B a tree is measured exactly when its column is picked, which has chance \displaystyle \frac{1}{20}. So the first condition holds for both.
Now look at groups. Plan B can only produce 20 different samples, one per column. A group with trees from two columns, like the scattered group in Figure 2, has chance 0. Plan B fails the second condition, so it is not a simple random sample; picking whole groups at random is a cluster sample.
It matters. Trees in one column share soil and light. If Plan B picks the stream column, all ten measured trees are the best-watered in the park. Plan A’s ten trees usually come from many columns, so no single stretch of ground decides the result.
Day 2: the council’s traffic survey
A made-up town has 8{,}000 adults. Its council wants their views on a new crossing over the bypass and draws 340 adults at random from the electoral roll, the list of adults registered to vote, which lists 6{,}800 of them. The 1{,}200 not on it include most residents of Larkfield, a new estate beside the bypass built last year, who have not yet registered. Name the population and the frame, find the percent of adults with no chance, find the chance for an adult on the roll, and judge the plan.
SolutionThe population is the town’s 8{,}000 adults; the frame is the 6{,}800 on the roll. The adults off the roll have chance 0, and they are
of the town’s adults. An adult on the roll has chance \displaystyle \frac{340}{6800} = \frac{1}{20} = 5\%.
The draw is a fair simple random sample of the roll, so it speaks for the roll. The missing adults are not a random handful: Larkfield residents live beside the bypass and may well want the crossing more than others, so the survey is likely to show less support than the town feels. Drawing 680 would not help; Larkfield would still have no chance. The repair is to the frame: add the Larkfield residents, for example from records of who moved into the new homes, and draw from the new list.
Day 2: the library’s card list
A town wants to know how many books a month its readers read. The library offers to draw 100 cards at random from its list of 5{,}000 library cards. 400 of the cards belong to people who have moved away, and a few people hold two cards after losing one. Name the three ways this frame falls short, find the average number of moved-away cards in the sample, and say what to do about each fault.
SolutionIt misses members: readers with no card, who buy books or read on a screen, have chance 0, and they may read more or less than card holders. The list is a fair frame for the library’s card holders and a poor one for the town’s readers.
It holds entries that are not members: the moved-away cards are \displaystyle \frac{400}{5000} = 0.08 = 8\% of the list. Every card has the same chance, so averaged over all the possible samples the sample holds
of them. Each is set aside and replaced by another card drawn at random from the list.
It lists some members twice, giving them two chances. Removing the older card of each pair before the draw restores one entry per person.
Day 2: a sample drawn perfectly that still misleads
Rhiannon-Mai, the orchestra’s secretary, draws 5 of its 30 members from a hat with every precaution from Example 1. Four of the five are string players. The chair maker’s salesperson then asks each of them, with the conductor listening: “These chairs are far better than your old ones, aren’t they?” All five say yes. What has random sampling made sure of, and what has it not?
SolutionString players are \displaystyle \frac{4}{5} = 80\% of the sample against \displaystyle \frac{12}{30} = 40\% of the orchestra. On average a sample of 5 holds \displaystyle 5 \times \frac{12}{30} = 2 string players, but one sample can land well away from its average, and this one did. Nothing went wrong in the draw; that is sampling variability.
The answers are another matter. The question tells the players what to say, and the conductor hears every reply: response bias, which chance cannot touch, since chance chose who was asked, not what they said. A neutral question answered on paper and collected by someone with no stake in the chairs would fix it.
Boundaries
It needs a frame. Without a list there is nothing to draw from. Nobody can list the fish in a lake or tomorrow’s passers-by; such populations need other methods.
It needs labels read fairly. With random digits every label has the same length and any block that is not a label is skipped, never bent to fit. With a calculator the range is exactly 1 to N.
It is drawn without replacement. If slips go back, a member can be chosen twice and the sample can hold fewer than n different members. That is a different process.
The promise is about the method, not the result. Nothing tilts a simple random sample, yet any one sample can be unlike the population, and it speaks only for its frame.
Where this goes next
The number a random sample gives is a statistic estimating a parameter, and the next question is how far it might miss: sampling variability. Other random designs build on this one. A systematic sample takes a random start and then every so many members down the list. A stratified random sample draws a simple random sample inside each of several groups. A cluster sample, like Plan B in Example 4, picks whole groups. Frames that miss people, and chosen people who never reply, are taken up in undercoverage and nonresponse.
Exercises
Day 1: chances and groups
- In a simple random sample, find each member’s chance of being chosen, as a fraction and a percent. (a) 8 of a club’s 40 members. (b) 10 of a park’s 200 trees. (c) 15 of a batch of 600 bottles.
- Two of five friends, Montague, Nell, Orlaith, Piers and Quenby, are chosen by drawing slips from a hat. (a) List every group of 2. How many are there? (b) What is each group’s chance? (c) How many groups include Nell, and what is her chance? Compare it with \displaystyle \frac{n}{N}.
- The orchestra’s 30 names go in a hat. Say what goes wrong in each case. (a) String players’ names are on small slips and everyone else’s on large cards. (b) The slips go in in alphabetical order, the hat is not shaken, and slips are taken from the top.
Day 1: drawing a sample
- A park’s 200 trees are numbered 001 to 200. From the first digit, read the row 73418 \quad 29060 \quad 57451 \quad 18200 \quad 01993 in blocks of three to choose 3 trees, and give the reason each skipped block is skipped.
- The orchestra’s members are numbered 01 to 30. From the first digit, use the row 47213 \quad 06603 \quad 21991 \quad 48250 to choose 4 members.
- A ranger’s calculator gives random whole numbers from 1 to 200 to choose 5 trees. They come out 57, 143, 57, 200, 9, 88, 131. Which trees are chosen, and why is 131 not used?
- Which of these are simple random samples of the orchestra’s 30 members? Explain each. (a) The conductor picks the 5 she thinks will be most honest. (b) The members are numbered and a calculator gives 5 different whole numbers from 1 to 30. (c) The first 5 to arrive at the next rehearsal. (d) The members sit in 6 sections of 5; a number cube is rolled and the whole section with that number is chosen.
- For plan (d) in exercise 7: (a) Find each member’s chance. (b) How many different samples can the plan produce? (c) There are 142{,}506 different groups of 5 that can be made from 30 people. What fraction of them can plan (d) produce?
Day 2: sampling frames
- Name the population, the frame and one group the frame misses. (a) A school wants parents’ views on a new starting time and draws from the parents signed up for its email newsletter. (b) A ranger wants to measure the trees now in a park and numbers them from a tree map drawn five years ago.
- A made-up village has 2{,}000 households, and its list of household email addresses has 1{,}500 of them. A simple random sample of 60 households is drawn from the list. (a) What percent of the households has no chance? (b) What is each listed household’s chance?
- A park’s tree map lists 200 trees. Since it was drawn, 10 of those trees have been felled and 10 new trees planted, so the park still has 200. (a) With the map as the frame, what percent of today’s trees has no chance? (b) 10 entries are drawn at random from the map. On average, how many are felled trees, and what should the ranger do when one is drawn?
- The orchestra’s list was made by joining the section lists. Montague plays violin and viola and is on both, so the list has 31 entries for 30 members. Five entries are drawn at random. Is this a simple random sample of the members? Who is favored, and what is the fix?
Day 2: what chance cannot fix
- 50 of a park’s 200 trees are oaks. A simple random sample of 10 trees includes 5 oaks. (a) Find the percent of oaks in the sample and in the park. (b) On average, how many oaks does a random sample of 10 trees hold? (c) Does the sample show it was drawn wrongly?
- The council in Example 5 asks its chosen adults: “Do you agree that dangerous speeding on the bypass makes a new crossing urgent?” Explain what the random draw has made sure of, and what it has not.
- Of the council’s 340 chosen adults, 119 send the survey back. (a) What percent replied? (b) A councillor suggests putting the survey online for anyone to fill in, to make up the numbers. Explain why that would not repair the sample, and suggest something better.
Reasoning
- Piers chooses members by closing his eyes and pointing at the printed list of 30 names. Explain why this is not a simple random sample.
- Quenby wants to waste fewer digits choosing from members 01 to 30. She uses every pair from 01 to 90: 31 to 60 stand for members 1 to 30 after taking away 30, and 61 to 90 after taking away 60; she skips 91 to 99 and 00, and skips repeats as usual. Explain why every member is still equally likely. Would it stay fair if she also used 91 to 99 for members 1 to 9?
- Orlaith says: “My sample was chosen at random, so its 60\% is exactly the population’s percent.” Explain what is right and what is wrong in this.
- Explain why drawing slips from a hat and putting each back before the next draw does not give a simple random sample.
Answers
- (a) \displaystyle \frac{8}{40} = \frac{1}{5} = 20\%. (b) \displaystyle \frac{10}{200} = \frac{1}{20} = 5\%. (c) \displaystyle \frac{15}{600} = \frac{1}{40} = 2.5\%. Each is \displaystyle \frac{n}{N}.
- (a) 10 groups; by first letters MN, MO, MP, MQ, NO, NP, NQ, OP, OQ, PQ. (b) No pair is favored, so each has chance \displaystyle \frac{1}{10}. (c) Nell is in 4 of them, so her chance is \displaystyle \frac{4}{10} = \frac{2}{5}, which is \displaystyle \frac{n}{N} with n = 2 and N = 5.
- (a) Large cards are easier to grab, so players who are not string players are favored. (b) The names put in last, late in the alphabet, lie on top and are the most likely to be drawn.
- Trees 057, 182 and 199. Blocks: 734 skip (above 200), 182 pick, 906 skip (above 200), 057 pick, 451 skip (above 200), 182 skip (already chosen), 000 skip (no tree 000), 199 pick.
- Members 03, 14, 21 and 30. Pairs: 47 skip, 21 pick, 30 pick, 66 skip, 03 pick, 21 skip (already chosen), 99 skip, 14 pick.
- Trees 9, 57, 88, 143 and 200. The second 57 is a repeat and is ignored; after 88 there are already 5 trees, so 131 is not needed.
- Only (b). (a) Her judgment decides, so her preferences shape the sample. (b) Chance alone decides and every group of 5 is equally likely. (c) A convenience sample: early arrivers may differ from late ones, and chance plays no part. (d) Every member has the same chance, but only 6 groups can ever be chosen, so the second condition fails.
- (a) \displaystyle \frac{1}{6}, the same as \displaystyle \frac{5}{30}. (b) 6, one per section. (c) \displaystyle \frac{6}{142506} = \frac{1}{23751}: one group in every 23{,}751.
- (a) Population: the school’s parents; frame: the newsletter list; missed: parents who never signed up, perhaps those without email. (b) Population: the trees in the park now; frame: the old map; missed: trees planted since it was drawn.
- (a) \displaystyle \frac{500}{2000} = 0.25 = 25\%. (b) \displaystyle \frac{60}{1500} = \frac{1}{25} = 4\%.
- (a) The 10 new trees are not on the map: \displaystyle \frac{10}{200} = 5\% of the 200 - 10 + 10 = 200 trees. (b) On average \displaystyle 10 \times \frac{10}{200} = \frac{1}{2}, so often none and sometimes one. A felled tree is outside the population: set it aside and draw another entry at random from the map.
- No. Montague has two entries, so roughly twice anyone else’s chance, and if both are drawn the sample holds only 4 different members. Remove one of his entries first, leaving 30 entries, one per member.
- (a) Sample: \displaystyle \frac{5}{10} = 50\%; park: \displaystyle \frac{50}{200} = 25\%. (b) \displaystyle 10 \times \frac{50}{200} = 2.5 oaks. (c) No. A random sample can hold more of one kind than its share, and 5 oaks is well above 2.5 but possible. Only the method shows whether a sample was drawn fairly; one result cannot.
- The draw made sure no one chose who was asked: every adult on the roll had the same chance. It has not made the answers fair. Calling the speeding dangerous and the crossing urgent pushes people toward yes, which is response bias; a neutral question such as “Should the council build a new crossing over the bypass?” is needed.
- (a) \displaystyle \frac{119}{340} = 0.35 = 35\%. (b) Online answers come from whoever chooses to answer, a voluntary response sample, standing in for the 221 chosen adults who did not reply. Better to contact the non-repliers again, by letter, phone or a visit, so the answers still come from the random sample.
- Pointing is not a process with equal chances. A finger tends to land near the middle of the page rather than the top or bottom lines, so members in the middle are more likely to be chosen and some groups far more likely than others.
- Each member owns exactly three of the 90 pairs used: member 7 owns 07, 37 and 67; member 30 owns 30, 60 and 90. Every pair is equally likely, so a kept pair lands on a given member with chance \displaystyle \frac{3}{90} = \frac{1}{30}, the same for all 30. Each kept pair is a fair draw of one member, and with repeats skipped it works like slips from a hat, so every group is equally likely too. Adding 91 to 99 would give members 1 to 9 four pairs each and the rest three, so members 1 to 9 would be favored: not fair.
- Right: chance chose the sample, so nothing tilted it, and its percent is right on average. Wrong: random samples vary, so 60\% is an estimate that may be some way from the population’s percent, not exactly equal to it.
- With each slip put back, a member can be drawn twice, so the sample may hold fewer than n different members. A simple random sample is drawn without replacement and always holds n different members, with every such group equally likely.