Statistics · S.IC.B.5

Control groups, placebos and blinding

People in a trial get better whether or not the new treatment works. How can an experiment tell how much of the improvement the treatment caused?

You need

Units, treatments, factors and levelsStatistics

Unlocks

Random assignment and confoundingStatistics

Where you are 7 in Experimental design

Statistics7

Sampling and study design6

Experimental design7

Comparing two treatments by re-randomizing

Control groups, placebos and blinding

Units, treatments, factors and levels Random assignment and confounding Completely randomized design and replication ▾ Show all 7 (2 more)

Why this exists

A sore throat almost always clears on its own within a week or so. Suppose 90 people suck a new throat lozenge and their throats clear in 4.4 days on average. Did the lozenge do that? The number cannot say. Their throats would have cleared anyway, and people who believe they have been treated often feel better sooner for that reason alone. A result about one group has nothing to be measured against.

An experiment imposes treatments on experimental units (the subjects, here) and compares their responses. This lesson is about what the treatment group is compared WITH, and about keeping that comparison honest: the control group, the placebo it often receives, and blinding, which keeps knowledge of who got what from leaking into the results.

Almaz, Rahel and Kalkidan run a made-up trial. They recruit 180 volunteers who have just come down with a sore throat. By chance, 90 get the new lozenge and 90 get a placebo lozenge that looks, tastes and is packed exactly like it but contains no active ingredient. Each volunteer sucks one lozenge four times a day, and the response is the number of days until the sore throat clears. All data in this lesson are made up.

Definition

Part 1. A control group is a group in an experiment that does not receive the treatment being tested, so that its response gives the baseline the treatment group is compared against. It can receive:

no treatment at all;
a placebo, a dummy treatment that looks and is given exactly like the real one but has no active ingredient (a sugar pill, a saline injection, a lozenge with no medicine in it);
the usual treatment, the method already in use, when the question is whether the new one is better than what people get now.

The control group is a treatment group like any other, chosen to answer a particular question. Its members are assigned in the same way as everyone else’s, by chance (Random assignment and confounding).

The placebo effect is a real change in the response that comes from receiving something believed to be a treatment, rather than from what the treatment contains. Expectation, extra attention and the routine of taking something can all produce it.

Part 2. An experiment is blind when people involved do not know which treatment each subject receives.

Single-blind: the subjects do not know which treatment they get.
Double-blind: the subjects do not know, and neither do the people who give the treatments, care for the subjects or measure the response.

Someone always knows: a person who holds the list linking each subject to a treatment. In a double-blind trial that person has no contact with the subjects and no part in measuring them.

Intuition

Every volunteer in the trial is pushed toward recovery by several things at once: the body fighting the infection, the belief that a lozenge is helping, a soothing coating on the throat, and, in one group only, the active ingredient. A placebo control receives every one of these except the ingredient. So when the two groups are compared, everything they share cancels, and the difference left over is what the ingredient did.

Figure 1 shows the trial’s results. Both groups got better; nobody was still sick on day 9. The lozenge group’s distribution sits about a day to the left of the placebo group’s. That shift, not the lozenge group’s own average, is the evidence.

Figure 1Days until the sore throat cleared in the two groups (made-up data). Both groups recover; the placebo group’s bars sit about a day further right.

Blinding protects the same comparison. A volunteer who knew they had the placebo would lose the belief the placebo is there to supply. A nurse who knew who had the real lozenge might, without meaning to, read a borderline throat as “clear” a day sooner for them. Either knowledge puts back a difference between the groups that the design had removed.

Results

The treatment effect is a difference. The estimated effect of a treatment is the difference between the treatment group’s mean response and the control group’s:

\text{estimated effect} = \bar{x}_{\text{treatment}} - \bar{x}_{\text{control}}.

Which effect it estimates depends on the control. Against a placebo it is the effect of the active ingredient; against the usual treatment it is the gain over current practice; against no treatment it is the effect of everything the treatment group received, placebo effect included.

Splitting the effect. With three groups, none, placebo and treatment, the full difference between none and treatment splits into two parts:

\begin{aligned} \bar{x}_{\text{none}} - \bar{x}_{\text{trt}} &= (\bar{x}_{\text{none}} - \bar{x}_{\text{plac}}) \\ &\quad + (\bar{x}_{\text{plac}} - \bar{x}_{\text{trt}}), \end{aligned}

the placebo effect plus the ingredient effect.

Blinding. In a blind experiment, knowing the treatment cannot change how subjects behave or report. In a double-blind experiment, it also cannot change how they are treated or measured.

Why it is true

Why the difference isolates the ingredient. Random assignment makes the two groups alike, apart from chance, in everything the volunteers bring with them: age, health, how bad the throat was. The placebo makes them alike in everything the trial does to them: the lozenge, the schedule, the visits, the belief that they might be treated. The one thing left that differs on purpose is the active ingredient. So any difference in mean response beyond what chance would produce belongs to the ingredient. Whether a particular difference is bigger than chance can produce is the question of Comparing two treatments by re-randomizing.

Why the split adds up. The identity in the Results is algebra: on the right, -\bar{x}_{\text{plac}} and +\bar{x}_{\text{plac}} cancel, leaving the left side. Its meaning comes from the design: the none and placebo groups differ only in receiving a dummy lozenge, and the placebo and treatment groups differ only in the ingredient.

Why the measurer must be blind too. A response like “the day the throat is clear” needs a judgment. If the person judging knows the group, that knowledge is a second difference between the groups, besides the ingredient, and nothing in the comparison can separate the two.

Worked examples

The lesson is taught in two parts. Examples 1 to 3 belong to the first: what the control group is for and the placebo effect in numbers. Examples 4 to 6 belong to the second: blinding, and what to do when it cannot be done.

Example 1

The trial’s two groups

The table gives the number of volunteers whose sore throat cleared after each number of days, in the trial of 180 volunteers. Find each group’s mean, and the difference. What does the difference estimate?

days	new lozenge	placebo
2	6	2
3	17	9
4	26	18
5	23	24
6	12	20
7	6	11
8	0	6

Solution

Each count says how many volunteers took that many days, so the total number of days in a group is the sum of days times count. For the lozenge group that total is 2(6) + 3(17) + 4(26) + 5(23) + 6(12) + 7(6) = 396 days, over 90 volunteers:

\bar{x}_L = \frac{396}{90} = 4.4 \text{ days}.

For the placebo group, 2(2) + 3(9) + 4(18) + 5(24) + 6(20) + 7(11) + 8(6) = 468 days:

\bar{x}_P = \frac{468}{90} = 5.2 \text{ days}.

The difference is 5.2 - 4.4 = 0.8 days. Both groups recovered, and the placebo group in about five days, so “the lozenge group recovered in 4.4 days” alone would have credited the lozenge with all of it. Since the groups differ only in the ingredient, the 0.8 days is the estimate of the ingredient’s effect: on average, the throat cleared 0.8 days sooner with it.

Example 2

Choosing the control

For each question, say what the control group in a trial of the new lozenge should receive.

(a) Does the lozenge’s medicine do anything?

(b) Should pharmacies stock the new lozenge instead of the honey lozenge they sell now?

Solution

(a) A placebo lozenge. The question is about the ingredient, so the control must get everything else: a lozenge of the same look, taste and schedule. This is the design of Example 1.

(b) The usual treatment, the honey lozenge. A placebo control would show whether the new lozenge beats nothing, but buyers already have the honey lozenge; the useful comparison is with it. When a treatment that works already exists, a placebo or no-treatment control can also be unfair to sick subjects, which is why trials of serious illnesses compare a new drug with the current one.

(c) No treatment. The question includes the effect of sucking something and of expecting relief, so the control must have neither.

The same lozenge, three questions, three controls: a control group is chosen for the comparison it makes, not because “control” means “nothing”.

Example 3

Separating the placebo effect

Rahel runs a second trial with 150 volunteers, 50 in each of three groups: no lozenge, placebo lozenge, new lozenge. The mean days to clear are 6.0, 5.1 and 4.3. Find the placebo effect, the ingredient effect and the full difference between no lozenge and the new lozenge.

Solution

The no-lozenge and placebo groups differ only in receiving a dummy lozenge, so the placebo effect is

6.0 - 5.1 = 0.9 \text{ days}.

The placebo and new-lozenge groups differ only in the ingredient, so the ingredient effect is 5.1 - 4.3 = 0.8 days. The full difference is 6.0 - 4.3 = 1.7 days, and indeed 0.9 + 0.8 = 1.7 (Figure 2).

Figure 2Mean days to clear in a three-group trial (made-up data). The gap between “nothing” and “new lozenge” is two effects added together.

A trial of the new lozenge against no lozenge would have reported 1.7 days and credited all of it to the medicine, when more than half of it, 0.9 of the 1.7 days, comes from taking any lozenge. The placebo group is what exposes this.

Example 4

Who must not know

In the trial of Example 1, Kalkidan packs the lozenges in bottles marked only with a code number, Rahel keeps the list of which code is which, and Almaz, a nurse, examines each volunteer’s throat every morning and records the day it is clear. Who must not know the groups, and why? Then suppose Almaz did know, and for 18 of the 90 lozenge volunteers she recorded a borderline throat as clear one day early. What difference would the trial report?

Solution

The volunteers must not know, or the placebo group loses the expectation the placebo supplies. Almaz must not know, because “clear” is her judgment. Kalkidan and Rahel know the code, but neither examines anyone. This is a double-blind design.

If 18 volunteers each lose one day, the lozenge group’s total drops by 18 days:

\bar{x}_L = \frac{396 - 18}{90} = \frac{378}{90} = 4.2 \text{ days}.

The trial would report 5.2 - 4.2 = 1.0 days instead of 0.8. The extra 0.2 days is 0.2 / 0.8 = 1/4 of the real effect, manufactured by the measurer, and nothing in the data would show it.

Example 5

Checking whether the blind held

Suppose the new lozenge had a strong cooling taste the placebo lacked. At the end, each volunteer is asked which lozenge they think they had: 81 of the 90 lozenge volunteers say “the real one”, and so do 18 of the 90 placebo volunteers. What does that say?

Solution

The proportions guessing “real” are 81/90 = 0.9 in the lozenge group and 18/90 = 0.2 in the placebo group. If the volunteers could not tell, guessing would be a coin toss in both groups and the two proportions would be about equal. They differ by 0.7, so most volunteers knew their group: the blind failed.

Then the 0.8-day difference no longer isolates the ingredient. Volunteers who knew they had the real thing expected more, and expectation is exactly the placebo effect. The fix is in the placebo, not in the analysis: give the placebo the same cooling taste.

Example 6

When the subjects cannot be blinded

A community center wants to know whether a new juggling app teaches beginners better than its usual weekly class. It assigns 40 beginners by chance, 20 to each, and after four weeks records the longest run of catches each can make with three balls. Can the study be blind? What should it do instead?

Solution

No subject can be blinded: everyone knows whether they went to a class or used an app, and there is no “placebo class”. The coaches cannot be blinded either. The design can still protect the comparison in other ways.

Use the usual method as the control. Both groups learn in some way, so both have a reason to expect progress; comparing the app with no instruction would mix in the effect of being taught at all.
Blind the measurer. Each attempt is filmed, and the runs of catches are counted by a judge who sees the videos without knowing who used what.
Choose a response that needs little judgment. A count of catches leaves less room for bias than a rating of “juggling skill”.
Keep everything else equal and describe both arms the same way. Equal practice time, and both groups told only that two ways of learning are being compared, so neither is told it has the exciting new one.

Such a study is not blind, and its report should say so, but the response is measured blind.

Boundaries

A control group compares only what it differs in. If the control group also differs in something besides the treatment (seen by a different doctor, recruited in a different month), the difference measures both at once; that is confounding, and random assignment is how an experiment avoids it.

Effects are relative to the control. “Clears 0.8 days sooner” means sooner than the placebo. The same lozenge compared with nothing, or with a honey lozenge, has a different effect.

A placebo exists only where a dummy can be made. Surgery, exercise, teaching methods and diets mostly have no exact fake, and a dummy version can be unethical when it carries risk with no benefit.

A blind that leaks is not a blind. A taste, a side effect or a label can tell subjects or measurers the group, and then the trial is blind on paper only.

A difference in sample means can be chance. With any finite group, two treatments with no real difference still give slightly different means. Judging whether 0.8 days is more than chance needs Comparing two treatments by re-randomizing.

Blinding does not remove all bias. Subjects can still give answers shaped by how the question is put (Response bias and question wording), and dropouts can differ between groups.

Where this goes next

A control group is the first of the principles of Experimental design. Its comparison is fair only if the groups start out alike, which Random assignment and confounding arranges by chance. Completely randomized design and replication adds replication, many units per treatment, so chance differences shrink, and Blocking: randomized block and matched pairs designs first groups subjects who are alike, for example by how bad the throat was on day one. Deciding whether a treatment-minus-control difference is real is the work of Comparing two treatments by re-randomizing.

Exercises

Exercises 1 to 8 go with the first part (Examples 1 to 3), Exercises 9 to 16 with the second (Examples 4 to 6), and the reasoning group draws on both.

Part 1: choosing the control

For Exercises 1 to 3, name the control group and say whether it gets no treatment, a placebo or the usual treatment.

A dentist gives 30 patients a new numbing gel after a tooth is pulled and 30 patients a gel that looks the same but contains no numbing drug, then records each patient’s pain score.
A delivery company gives half its drivers a new route-planning app and the other half keep the paper maps they already use; it records each driver’s daily miles.
A swim coach has half her swimmers add a new kicking drill to practice; the other half practice as before with no drill added. She records each swimmer’s time over 50 meters.
A new asthma inhaler is to be tested, and an inhaler that works well already exists. Explain why the control group should get the existing inhaler rather than a placebo.

Part 1: the placebo effect in numbers

A company tests a second lozenge on 80 volunteers, 40 per group. Days until clear for the lozenge group: 3 days, 10 people; 4 days, 14; 5 days, 10; 6 days, 6. For the placebo group: 3 days, 4 people; 4 days, 10; 5 days, 14; 6 days, 8; 7 days, 4. Find both means and the estimated effect of the lozenge.
A three-group trial of a throat spray gives these mean days to clear: no spray 6.2, placebo spray 5.6, real spray 5.5. Find the placebo effect, the ingredient effect and the full difference. What share of the full difference is the placebo effect?
Using the table of Example 1, find the proportion of each group whose throat cleared within 4 days, and the difference.
A trial compares the new lozenge with the honey lozenge pharmacies sell, 90 volunteers each. Mean days to clear: new 4.6, honey 4.9. Find the estimated effect, and say what the trial cannot tell you.

Part 2: who is blinded

In a trial of a sleep supplement against a placebo capsule, subjects write down how many hours they slept, and a researcher who knows the groups collects the diaries. Is the trial single-blind, double-blind or neither, and does it matter here?
Two orange juices are compared in a taste test. Describe a way to make it double-blind.
A trial compares knee surgery with a program of physical therapy. Can the patients be blinded? Can the person who measures how far each knee bends?
In a throat spray trial of 120 volunteers, 60 per group, 33 of the spray group and 29 of the placebo group guess at the end that they had the real spray. Find both proportions. Does the blind seem to have held?

Part 2: bias and the impossible blind

In the trial of Example 1, suppose an unblinded nurse recorded the clearing day one day late for 27 of the 90 placebo volunteers. Find the placebo mean she would report, the difference the trial would report, and how much of it is bias.
An office tests whether standing desks improve focus, with 30 workers assigned by chance to standing desks and 30 to their usual desks. Focus is a supervisor’s rating. Explain why the workers cannot be blinded, and suggest two ways to protect the comparison.
A study assigns half of 60 volunteers to a 20-minute daily walk and half to their usual routine; a wrist sensor records hours of sleep. Which parts of the study can be blind?
Rahel proposes giving the no-lozenge group of a trial a glass of water four times a day. What would that add to a no-treatment control, and what would it still not do?

Reasoning

Explain why a placebo lozenge must match the real one in look, taste, packing and schedule, not just in having no medicine.
In Example 1 both groups recovered. Explain why this does not mean the lozenge failed, and why it does not mean the lozenge worked either.
Explain why a trial whose control group gets no treatment at all cannot be blind for the subjects.
Explain why blinding the measurer matters less when the response is a thermometer reading than when it is “the day the throat is clear”, and why it still matters at all.
A volunteer complains that the placebo group is wasted, since its members get no medicine. Explain what the trial would lose without it.

Answers

The 30 patients given the gel with no numbing drug; a placebo.
The drivers who keep the paper maps; the usual treatment.
The swimmers who practice as before; no treatment (no added drill).
The question doctors face is whether the new inhaler beats the one patients already use, so that is the useful comparison. And withholding a treatment that works from people with asthma, by giving them a placebo, would put them at risk for no benefit.
Lozenge: 3(10) + 4(14) + 5(10) + 6(6) = 172, so 172/40 = 4.3 days. Placebo: 3(4) + 4(10) + 5(14) + 6(8) + 7(4) = 198, so 198/40 = 4.95 days. Estimated effect: 4.95 - 4.3 = 0.65 days sooner with the lozenge.
Placebo effect 6.2 - 5.6 = 0.6 days; ingredient effect 5.6 - 5.5 = 0.1 days; full difference 6.2 - 5.5 = 0.7 days. Share: 0.6/0.7 \approx 0.86, so most of the spray’s apparent benefit is the placebo effect.
Lozenge: 6 + 17 + 26 = 49, and 49/90 \approx 0.544. Placebo: 2 + 9 + 18 = 29, and 29/90 \approx 0.322. Difference 20/90 \approx 0.222: about 22 percentage points more of the lozenge group cleared within 4 days.
4.9 - 4.6 = 0.3 days sooner with the new lozenge. Without a placebo or no-treatment group, the trial cannot say whether either lozenge beats nothing.
Single-blind: the subjects do not know, but the researcher does. It matters less than usual because the subjects, who are blind, do the measuring; the researcher should still not handle or interpret the diaries with the groups in view.
One person pours the juices into cups marked only with codes and keeps the code list; a second person, who does not know the codes, serves the cups and records the tasters’ ratings. The tasters do not see the cartons.
The patients cannot be blinded: they know whether they had surgery. The person measuring the bend can be, if patients are asked not to say which treatment they had and scars are covered.
33/60 = 0.55 and 29/60 \approx 0.483. The two are close, and both near a half, so the guesses look like chance: the blind seems to have held.
Placebo mean (468 + 27)/90 = 495/90 = 5.5 days. Reported difference 5.5 - 4.4 = 1.1 days, of which 1.1 - 0.8 = 0.3 days is bias.
A worker can see whether the desk is a standing desk. Protect the comparison by having focus measured by someone who does not know the desk type (or by a count, such as tasks finished, rather than a rating), and by describing both arms to workers in the same neutral way.
The volunteers cannot be blind: they know whether they walk. The sleep measurement is made by a sensor, and the person analyzing the sensor data can be blinded to the groups.
It gives the control group a routine and something to do four times a day, so some of the attention and routine of the treatment is matched. It is not a placebo: the volunteers know water is not a lozenge, so it supplies no belief of being treated, and the trial is not blind.
Any difference between the real and dummy lozenge besides the medicine can tell subjects their group, which breaks the blind, and can itself affect the throat (a soothing coating, a cooling taste). The only difference allowed is the one being tested.
Recovery was expected in both groups, since sore throats clear on their own; the question is whether the lozenge group recovered faster, and it did by 0.8 days. But a gap in sample means could also be partly chance, so recovery in both groups proves neither success nor failure.
Subjects who receive nothing know they have received nothing. Being blind requires something that could be the treatment, which is what a placebo supplies.
A thermometer gives a number with little room for judgment, so the measurer’s expectations have less to work on. But a measurer who knows the groups can still retake a reading, choose when to measure, or treat the subjects differently, so blinding still protects the comparison.
Without it there is no baseline: the lozenge group’s 4.4 days could be ordinary recovery plus the placebo effect. The placebo group’s 5.2 days is what shows the medicine added 0.8 days.