Math 6

Collecting data with a survey

You want to know what a whole class thinks, so you ask them. What makes the question you ask, and the way you write down the answers, good enough to trust?

You need

Statistical questions and variabilityMath 6

Unlocks

Response bias and question wordingMath 7

Where you are 8 in Data and variables

Statistics7

Data and variables8

Collecting data with a survey

Data tables, tallies and frequency tables Individuals, variables and units Sorting objects into groups Statistical questions and variability ▾ Show all 8 (3 more)

When you reach for it

A statistical question such as “Which dish on the cafeteria menu do the students in our class like best?” expects a different answer from each student, and the answers are in the students’ heads. To get them you survey the students: ask each one the same question, in the same words, and write down every answer. Each person is an individual and each answer is one value of the variable asked about.

The lesson takes two days. Part 1 plans the survey: the question and its choices, who to ask, and how to record and count the answers. Part 2 looks at wording, because the same people asked about the same thing in different words can give different data.

The procedure

Part 1: the question each person is asked

The statistical question is about the group. The survey question is put to each person, so it is about that person: “Which dish on the cafeteria menu do you like best?” All the answers together answer the statistical question.

A survey question is open when people answer in their own words, and fixed-choice when they pick from a list. Open answers must be sorted into groups before counting, and the sorter has to decide whether “noodles” and “pasta” are the same answer. Fixed choices are already the groups, so counting is quick, but only if every possible answer fits exactly one choice:

No gaps. Every truthful answer has a choice. A last choice such as “something else” catches what the list cannot name.
No overlaps. No answer fits two choices, or two people with the same answer can tick different boxes and the counts stop meaning anything definite.
One answer each (“choose one”), unless the question is meant to allow several.

When the answers are numbers, the question names the unit (“in dollars”, “days last week”), or one person’s 2 means hours and another’s means kilometers. Choices for numbers are pieces of the number line, and the two rules say the pieces must meet end to end without sharing a point. How to write the ends depends on the kind of number (Types of data: categorical, numerical, discrete, continuous):

For counts, which are whole numbers, “0 to 4” and “5 to 9” leave no gap, because no whole number lies between 4 and 5.
For measurements and money, an answer can be 4.5, so the ends are stated exactly: “under 5”, “5 to under 10”. “Under 10” means less than 10, so 10 belongs to the next piece.

Figure 1Each answer choice is a piece of the number line. In the draft, 5 and 10 belong to two pieces and amounts over 15 to none. In the repair the pieces meet end to end; a filled dot means that amount is in the piece, an open dot means it is not.

Part 1: who to ask

Ask the group the statistical question is about. A small group, such as one class, can be asked in full, and the answers then describe it exactly. For a group too big to ask everyone, the people asked must be like the whole group, and choosing them is the work of Populations, samples and census. Either way, ask each person once and note how many were asked.

Part 1: recording and counting

Write each answer down as it is given, or make one tally mark per answer, so nothing is lost or counted twice. A frequency table then lists each choice beside its frequency, the number of people who gave it. When each person gave one answer, there is a free check:

\text{sum of the frequencies} = \text{number who answered}

Each choice can then be given as a share of the people who answered, as a fraction, a decimal or a percent (Relative frequency tables builds whole tables of these):

\text{share} = \frac{\text{frequency}}{\text{number who answered}}

Part 2: how a question is worded

Four faults in the words change what people answer, or make the answers impossible to read.

A leading question tells people which answer is wanted: “Don’t you agree the new fish pie is delicious?” Saying no now means contradicting the asker.
An unclear word means different amounts to different people: in “Do you often play tag?”, one person’s “often” is daily, another’s weekly. Use a number and a time instead: “On how many of the last 5 school days did you play tag?”
Two questions in one: “Do you like the soup and the salad?” A “yes” might mean both or one. Ask two questions.
Choices with gaps or overlaps, as in Part 1.

A neutral question names its topic without praising or blaming any answer, asks one thing, uses words that mean the same to everyone, and offers balanced choices.

Wording changes the frequencies, and every fraction and percent is built from them, so a wording fault passes into every share, however many people are asked. Why people may answer away from the truth even when the words are fair is the subject of Response bias and question wording.

Worked examples

Example 1

Part 1: a question for each student

Bettany wants to answer “Which of the main dishes on this term’s cafeteria menu do the students in my class like best?” The dishes are pasta bake, chicken wrap, vegetable curry and fish pie. A first draft of the survey question reads “What’s your favorite food?” Explain what is wrong with it and write a fixed-choice question that works.

Solution

The draft is open and about all food, so it can return “ice cream”, which says nothing about the menu. A question about the menu that offers its dishes:

“Which main dish on the cafeteria menu do you like best? Choose one: pasta bake, chicken wrap, vegetable curry, fish pie, I have not tried any of them.”

Every student fits exactly one choice: “choose one” means a single dish, and a student who never eats in the cafeteria fits only the last choice, which fills the gap the four dishes would leave.

Example 2

Part 1: tallying the cafeteria answers

Bettany asks all 25 students in her class and writes down the answers in order (made-up data; “none” means “I have not tried any of them”):

pasta, chicken, pasta, curry, fish, chicken, pasta, none, curry, chicken, pasta, chicken, fish, pasta, curry, chicken, pasta, none, curry, chicken, pasta, fish, curry, chicken, pasta.

Make a frequency table with each choice as a fraction and a percent of the class, and answer the statistical question.

Solution

Going through the list once, one tally mark per answer:

Choice	Tally	Frequency	Share
pasta bake	\cancel{\mathsf{IIII}}\;\mathsf{III}	8	\displaystyle \frac{8}{25} = 32\%
chicken wrap	\cancel{\mathsf{IIII}}\;\mathsf{II}	7	\displaystyle \frac{7}{25} = 28\%
curry	\cancel{\mathsf{IIII}}	5	\displaystyle \frac{5}{25} = 20\%
fish pie	\mathsf{III}	3	\displaystyle \frac{3}{25} = 12\%
none	\mathsf{II}	2	\displaystyle \frac{2}{25} = 8\%
total		25	1 = 100\%

The frequencies add to 8 + 7 + 5 + 3 + 2 = 25, the number asked, so no answer was missed or counted twice. Each percent comes from multiplying top and bottom by 4, since 25 \times 4 = 100; for example \displaystyle \frac{8}{25} = \frac{32}{100} = 32\%.

Pasta bake is liked best, by 8 students. It is not the choice of most of the class: 32\% is less than half, and 25 - 8 = 17 students chose something else.

Example 3

Part 1: choices for amounts of money

For a field trip to a science museum, Bodo wants to know how much money the students in his class plan to spend in the museum shop. A first draft offers “0 to 5 dollars”, “5 to 10 dollars” and “10 to 15 dollars”. Find the amounts that fit two choices or none, and repair the list.

Solution

Picture each choice as a piece of the number line, as in the figure. The pieces share their ends: 5 dollars is in “0 to 5” and “5 to 10”, and 10 dollars is in “5 to 10” and “10 to 15”. Those are the overlaps. Every amount over 15 dollars, such as 18, is in no piece: the gap.

Money can be a decimal, such as 7.5 dollars, so the ends must be exact:

“How much do you plan to spend in the museum shop, in dollars? Choose one: under 5; 5 to under 10; 10 to under 15; 15 or more.”

Test the awkward amounts: 5 is not under 5, so it is only in “5 to under 10”; likewise 10 is only in “10 to under 15”, and 15 and 18 only in “15 or more”.

Example 4

Part 1: who to ask, and counting numerical answers

Bertram wants to answer “On how many of the last 5 school days did the students in my class play a ball game at recess?” for his class of 20. One plan is to ask the children on the ball court at the next recess; another is to ask all 20 classmates during the morning.

(a) Which plan should he use, and why?

(b) He asks all 20 and gets (made-up data) 3, 5, 0, 2, 3, 4, 5, 1, 3, 2, 0, 5, 3, 2, 4, 1, 3, 5, 2, 0. Make a frequency table, and find the fraction and percent of the class who played a ball game on at least 3 days, and on none.

Solution

(a) The second. The question is about his class, which is small enough to ask in full. The ball court misses classmates who are elsewhere and adds children from other classes; and everyone there is playing a ball game that day, so the answers would lean high.

(b) The answers count days, so they are the whole numbers 0 to 5, and those six values are themselves a list of choices with no gaps and no overlaps.

Days	Frequency
0	3
1	2
2	4
3	5
4	2
5	4

Check: 3 + 2 + 4 + 5 + 2 + 4 = 20.

“At least 3 days” means 3, 4 or 5: 5 + 2 + 4 = 11 students, and \displaystyle \frac{11}{20} = \frac{55}{100} = 55\%. “None” is the 0 row: \displaystyle \frac{3}{20} = \frac{15}{100} = 15\%.

Example 5

Part 2: finding and repairing wording faults

Name the fault in each draft and write a neutral question.

(a) “Don’t you think recess games are more fun when the teachers join in?”

(b) “Do you eat in the cafeteria regularly?”

(d) “How many recess games do you know the rules of? Choose one: 1 to 3; 3 to 6; 6 or more.”

Solution

(a) Leading: “Don’t you think” expects yes. Neutral: “Are recess games more fun with the teachers joining in, less fun, or about the same?”

(b) Unclear word: “regularly” could mean daily or weekly. Neutral: “On how many of the last 5 school days did you eat in the cafeteria?”, with choices 0 to 5.

(c) Two questions in one. A student who wants the farm but would rather buy lunch there has no honest answer. Ask “Where should our field trip go?” and, separately, “Would you rather take a packed lunch or buy lunch there?”

(d) Overlaps and a gap: 3 and 6 each fit two choices, and a student who knows no games fits none. The answers are whole numbers, so: “0; 1 to 3; 4 to 6; 7 or more.”

Example 6

Part 2: two wordings, two results

Blaise wants to know whether students would keep pizza as the Friday cafeteria dish. He draws names to split 40 students into two groups of 20, so the groups are alike, and asks each group a different question (made-up data).

Group A: “Do you want the cafeteria to keep pizza on Fridays?” 15 answer yes.

Group B: “Do you want the cafeteria to replace Friday pizza with a different dish each week, for more variety?” 12 answer yes.

Find the percent in each group who want to keep pizza, and explain the difference.

Solution

In group A, keeping pizza is a “yes”: \displaystyle \frac{15}{20} = \frac{75}{100} = 75\%.

In group B, a “yes” means replacing pizza, so keeping it is a “no”: 20 - 12 = 8 students, and \displaystyle \frac{8}{20} = \frac{40}{100} = 40\%.

The groups were made alike by drawing names, so the drop from 75\% to 40\% comes from the questions. Group B’s question gives one side a reason, “for more variety”, and the other side none, so it leans toward replacing. Each percent is the answer to its own wording. A balanced version: “On Fridays, would you rather the cafeteria serve pizza every week, or a different dish each week?”

Example 7

Part 2: what overlapping choices do to the counts

A survey asks 25 students “How many times have you visited a museum?” with the choices “never”, “1 to 2 times”, “2 to 5 times” and “more than 5 times”. Suppose their true numbers of visits are these (made-up data): 0 visits, 4 students; 1, 5; 2, 6; 3, 4; 4, 2; 5, 2; 6, 1; 9, 1.

(a) What are the smallest and largest numbers of students who could tick “1 to 2 times”, and what percents of the 25 are they?

(b) Repair the choices and give the frequencies they would produce.

Solution

(a) An answer of 1 fits only “1 to 2”, so those 5 students must tick it. An answer of 2 fits both “1 to 2” and “2 to 5”, so each of those 6 students could tick either:

\begin{aligned} \text{smallest} &= 5, & \tfrac{5}{25} &= 20\% \\ \text{largest} &= 5 + 6 = 11, & \tfrac{11}{25} &= 44\% \end{aligned}

The same students could produce any share from 20\% to 44\%, and the survey cannot tell which happened. “2 to 5” could likewise get anywhere from 4 + 2 + 2 = 8 to 8 + 6 = 14.

(b) Visits are whole numbers, so: “never; 1 or 2 times; 3 to 5 times; 6 or more times”. The frequencies are 4, then 5 + 6 = 11, then 4 + 2 + 2 = 8, then 1 + 1 = 2, and 4 + 11 + 8 + 2 = 25. As percents of 25: 16\%, 44\%, 32\% and 8\%.

Boundaries

A survey records what people say. Something that can be counted or measured directly is more trustworthy measured, and people can misremember or shade the truth even when the words are fair (Response bias and question wording).
The results describe the people who answered. Using them for a larger group needs the people asked to be like that group (Populations, samples and census).
The frequencies add to the number asked only when each person gives one answer. With “choose all that apply” they can add to more, and the percents to more than 100\%.
A fixed list works only when every answer can be foreseen. Otherwise ask an open question and decide how to sort the answers before counting them.

Where this goes next

A survey’s frequency table is where displays begin, starting with Relative frequency tables. Asking part of a big group so that it speaks for the whole is Populations, samples and census, and the ways answers drift from the truth are Response bias and question wording.

Exercises

Part 1: questions and choices

A draft asks “Which drink do you have with your cafeteria lunch? Choose one: milk, water, juice.” Describe a student these choices do not fit, and repair the list.
A recess survey asks “How many minutes did you spend on the climbing frame at recess yesterday? Choose one: 0 to 5; 5 to 10; 10 to 15.” Recess lasts 15 minutes. Which choices does an answer of 5 minutes fit? Of 10? Of 7.5 minutes? Repair the list.
A survey asks “How far do you live from school?” and gets “2”, “15”, “3 kilometers” and “half an hour”. Explain the trouble and rewrite the question.
For “How many field trips have you been on since you started school?”, which list fits every whole-number answer exactly once? (a) 0; 1 to 3; 3 to 6; more than 6. (b) 0; 1 to 2; 4 to 6; 7 or more. (c) 0; 1 to 3; 4 to 6; 7 or more.

Part 1: recording and counting

Borislav asked 20 classmates “Which game do you most like to play at recess?” with the choices tag, skipping, ball games, hopscotch and something else. In order, the answers were: tag, ball, skipping, tag, else, ball, hopscotch, tag, skipping, ball, tag, else, skipping, ball, tag, hopscotch, ball, else, skipping, tag. Make a frequency table with each choice as a fraction and a percent of the 20.
Borislav’s class of 32 voted on the field trip: farm 12, science museum 10, aquarium 6, castle 4. What fraction and percent chose the farm? How many other students would have had to vote for the farm instead for it to get more than half the votes?
Bodo asked 40 students “On how many of the last 5 school days did you eat the cafeteria lunch?” The frequencies for 0, 1, 3, 4 and 5 days were 6, 4, 9, 7 and 8; the one for 2 days was smudged. Find it, and the percent who ate the cafeteria lunch on at least 4 days.
Of the 30 students Bertram asked, 40\% chose ball games. How many students is that? Another 9 chose tag. What percent is that?
Bettany wants to know which cafeteria dish the whole school of 300 students likes best. One plan is to ask the students in the line for pasta bake on a pasta day. Explain why this will not do, and describe a better plan.

Part 2: finding the fault

For each of Exercises 10 to 14, name the fault and write a neutral question or list.

“Don’t you agree that the chicken wrap is the tastiest dish on the menu?”
“Do you play outside at recess a lot?”
“Should the field trip go to the farm and leave early in the morning?”
“Which is your favorite recess game? Choose one: ball games, soccer, tag, skipping.”
“How much would you pay for a field trip ticket, in dollars? Choose one: under 5; 5 to under 10; more than 15.”

Part 2: how wording changes the counts

Two alike groups of 25 students are asked about the field trip. Group A: “Would you like the field trip to be at the farm, where you can feed the animals?” 18 say yes. Group B: “Would you like the field trip to be at the farm, an hour’s bus ride away?” 9 say yes. Give each group’s percent of yes answers, compare them, and say which words push each way.
A survey asks 20 students “How many times did you buy a cafeteria snack last week? Choose one: 0 to 2; 2 to 4; 4 or more.” Their true numbers are: 0 times, 3 students; 1, 4; 2, 5; 3, 3; 4, 3; 5, 2. What are the smallest and largest numbers who could tick “2 to 4”, as counts and percents? Repair the choices and give the new frequencies and percents.
A survey asks 20 students “Do you often eat the cafeteria lunch?” and 12 say yes. The same 20 are then asked “On how many of the last 5 school days did you eat the cafeteria lunch?”, with frequencies for 0 to 5 days of 2, 3, 4, 5, 3, 3. How many, and what percent, ate it on at least 3 days? On at least 2 days? Why does the “often” count match neither?

Reasoning

Explain why, when each of n people gives one answer from a list with no gaps and no overlaps, the frequencies add to n and the percents to 100\%. What changes with “choose all that apply”?
A survey with a leading question is repeated with five times as many people. Explain why the larger count does not fix the problem.
Explain why “0 to 4; 5 to 9; 10 or more” works for “How many brothers and sisters do you have?” but not for “How many kilometers do you live from school?”, and repair it for the distance question.

Answers

A student who drinks nothing, or something else such as a smoothie, fits no choice. Repair: “milk, water, juice, something else, nothing”.
5 fits “0 to 5” and “5 to 10”; 10 fits “5 to 10” and “10 to 15”; 7.5 fits only “5 to 10”. Repair: “under 5 minutes; 5 to under 10; 10 to 15”. No answer exceeds 15, so every time fits exactly one choice.
There is no unit: “2” and “15” could be kilometers, miles or minutes, and “half an hour” is a time, not a distance. Rewrite: “How far do you live from school, in kilometers, to the nearest kilometer?”
Only (c). In (a), 3 fits two choices. In (b), 3 fits none. In (c) no whole number lies between one piece and the next.
Tag 6, \displaystyle \frac{6}{20} = 30\%; skipping 4, \displaystyle \frac{4}{20} = 20\%; ball games 5, \displaystyle \frac{5}{20} = 25\%; hopscotch 2, \displaystyle \frac{2}{20} = 10\%; something else 3, \displaystyle \frac{3}{20} = 15\%. Check: 6 + 4 + 5 + 2 + 3 = 20.
\displaystyle \frac{12}{32} = \frac{3}{8} = 37.5\%. Half of 32 is 16, so more than half means at least 17: 17 - 12 = 5 more students.
40 - (6 + 4 + 9 + 7 + 8) = 40 - 34 = 6. At least 4 days: 7 + 8 = 15, and \displaystyle \frac{15}{40} = \frac{3}{8} = 37.5\%.
0.4 \times 30 = 12 students. \displaystyle \frac{9}{30} = \frac{3}{10} = 30\%.
Everyone in that line has just chosen pasta bake, so it would come out far ahead. Better: names drawn from the whole school list (Populations, samples and census), or every student.
Leading: “Don’t you agree” expects yes. Neutral: “Which main dish on the menu do you find tastiest? Choose one: pasta bake, chicken wrap, vegetable curry, fish pie, I have not tried any.”
Unclear: “a lot” means different amounts to different people. Neutral: “On how many of the last 5 school days did you play outside at recess?”, choices 0 to 5.
Two questions in one: the place and the start time. Ask “Where should the field trip go?” and “At what time should the bus leave?” separately.
Overlap: soccer is a ball game, so a soccer fan fits two choices. Gap: hopscotch fits none. Repair: “soccer, other ball games, tag, skipping, something else”.
Gap: amounts from 10 up to and including 15 fit no choice. Repair: “under 5; 5 to under 10; 10 to under 15; 15 or more”.
A: \displaystyle \frac{18}{25} = 72\%. B: \displaystyle \frac{9}{25} = 36\%. A’s percent is 2 times B’s. “Where you can feed the animals” pushes toward yes, “an hour’s bus ride away” toward no. Neutral: name only the place, or describe every choice the same way.
Only the 3 students with 3 snacks must tick “2 to 4”: smallest 3, \displaystyle \frac{3}{20} = 15\%. The 5 with 2 and the 3 with 4 also could: largest 5 + 3 + 3 = 11, \displaystyle \frac{11}{20} = 55\%. Repair: “0 or 1; 2 or 3; 4 or more”, with frequencies 3 + 4 = 7, 5 + 3 = 8 and 3 + 2 = 5, which are 35\%, 40\% and 25\%.
At least 3 days: 5 + 3 + 3 = 11 students, 55\%. At least 2 days: 4 + 11 = 15 students, 75\%. The 12 yes answers are 60\%, matching neither, because each student decided alone what “often” means.
Each person is counted in exactly one row, so adding the rows counts everyone once: n. The shares all have denominator n, so they add to \displaystyle \frac{n}{n} = 1 = 100\%. With “choose all that apply” one person can sit in several rows, so the frequencies can add to more than n.
The words push every person the same way, so more people only make the percent settle more firmly on the answer to the leading question, not on what people think.
Brothers and sisters are whole numbers, and no whole number lies between 4 and 5 or between 9 and 10. A distance can be 4.5 kilometers, which fits no choice. Repair: “under 5; 5 to under 10; 10 or more”.