Statistics

Units, treatments, factors and levels

Before an experiment can compare anything, what exactly is being compared, on what, and how is the result measured?

Unlocks

Control groups, placebos and blindingStatistics

Where you are 7 in Experimental design

Statistics7

Sampling and study design6

Experimental design7

Comparing two treatments by re-randomizing Control groups, placebos and blinding

Units, treatments, factors and levels

Random assignment and confounding Completely randomized design and replication ▾ Show all 7 (2 more)

Why this exists

An experiment imposes something on purpose and measures what happens, which is what lets it show cause and effect (Observational studies and experiments). But “we tried fertilizer on some soybeans” is not yet a plan. Tried it on what: single plants, rows, whole fields? How much fertilizer, and compared with what? Was anything else changed at the same time? What was measured at the end?

Farrukh, Ismoil and Abebe are planning a soybean trial on a field split into small test plots. They want to know how fertilizer and watering affect the harvest. Before they can randomize anything or count anything, they need words for the parts of an experiment: the things that get treated, the conditions that are set, the possible settings of each, and the outcome. This lesson names those parts and shows one thing that testing two conditions together can reveal and testing them one at a time cannot. All data in it are made up.

Definition

The experimental units are the individuals an experiment assigns treatments to and measures: the smallest thing that can receive a treatment of its own. In the soybean trial each unit is a plot, a small rectangle of field, because fertilizer and water are given plot by plot. When the units are people, they are called subjects (or participants).

A factor is an explanatory variable that the experimenter sets. Each value a factor is set to in the experiment is a level of that factor. The trial has two factors:

fertilizer, with three levels: none, 40 pounds per acre and 80 pounds per acre;
watering, with two levels: light and heavy.

A treatment is one specific condition applied to a unit. With one factor, each level is a treatment. With several factors, a treatment is a combination of one level of each factor: “40 pounds per acre with heavy watering” is one treatment. Fertilizer by itself is not a treatment; it is a factor.

The response variable is the outcome measured on every unit after the treatment. Here it is the yield of each plot, in bushels per acre. The explanatory and response variables are the same roles met in plotting paired data; in an experiment the explanatory variables are the factors.

Intuition

Lay the levels of one factor across the top and the levels of the other down the side. Every box of the grid is one combination, so every box is one treatment. Figure 1 is the trial’s plan: 3 columns times 2 rows gives 6 treatments, and each treatment is given to 4 plots, so the trial needs 6 \times 4 = 24 plots.

Figure 1Two factors, fertilizer at three levels and watering at two, give six treatments. Each treatment goes to four plots, so the trial uses 24 plots.

Which 4 plots get which treatment is decided by chance; how that is done, and why, belongs to Random assignment and confounding. Using several units per treatment, rather than one, is the subject of Completely randomized design and replication.

Testing both factors in one grid shows something two separate experiments cannot. Figure 2 plots the mean yield of each treatment, one line per watering level. With light watering, going from 40 to 80 pounds of fertilizer lowers the yield; with heavy watering, the same step raises it. The effect of fertilizer depends on the watering. That dependence is called an interaction, and it shows on the graph as lines that are not parallel.

Figure 2Mean yield for each treatment (made-up data). The lines are not parallel, so the two factors interact.

Results

Counting treatments. If an experiment has factors with k_1, k_2, \ldots, k_m levels and uses every combination, it has

k_1 \times k_2 \times \cdots \times k_m \text{ treatments}.

If each treatment goes to n units, the experiment needs n times that many units.

Interaction. Two factors interact when the effect of changing one factor’s level depends on the level of the other factor. On a graph of treatment means, with one line per level of one factor, no interaction means parallel lines, and an interaction means lines that are not parallel (up to the chance variation in the means).

Only an experiment with both factors can see an interaction. An experiment that varies one factor while holding the other at a single level measures that factor’s effect at that one level only.

Why it is true

The count. Every fertilizer level can be paired with every watering level. Each of the 3 fertilizer levels gives 2 treatments, one per watering level, so there are 3 + 3, or 3 \times 2, treatments. A third factor with k_3 levels splits each of those again into k_3, and so on: this is the multiplication principle of counting.

The interaction. A one-factor fertilizer experiment run with light watering tests only the top row of Figure 1. It finds the fertilizer effect in that row and nothing else. Whether the bottom row behaves the same way is exactly the question of interaction, and that row was never grown. A separate watering experiment run with no fertilizer tests only the left column. Neither experiment contains the box “80 pounds with heavy watering”, so neither can discover what happens there. The full grid contains every box, so it can compare the fertilizer effect row by row.

Worked examples

Example 1

Naming the parts of the soybean trial

In the trial of Figure 1, name the experimental units, the factors and their levels, the treatments and the response variable.

Solution

The units are the 24 plots: fertilizer and water are applied to a plot as a whole, and the yield is measured plot by plot.

The factors are the two things the team sets: fertilizer, with levels none, 40 and 80 pounds per acre, and watering, with levels light and heavy.

The treatments are the 3 \times 2 = 6 combinations, numbered as below (fertilizer in pounds per acre across the top, watering down the side):

	none	40	80
light	1	2	3
heavy	4	5	6

The response variable is the yield of each plot, in bushels per acre. Each treatment is given to 4 plots, and 6 \times 4 = 24 plots in all.

Example 2

People as subjects

Abebe runs a second experiment, with people. He recruits 32 volunteers who have never tied a bowline, a sailing knot. Half learn it from a printed diagram and half from a short video; within each of those, half get three practice tries before the test and half get none. On the test, he times how many seconds each volunteer takes to tie the knot correctly. He also records each volunteer’s age. Name the parts of the experiment.

Solution

The experimental units are the 32 volunteers, and since they are people they are called subjects.

The factors are instruction format, with levels diagram and video, and practice, with levels none and three tries. Both are set by Abebe.

The treatments are the 2 \times 2 = 4 combinations: diagram with no practice, diagram with practice, video with no practice, video with practice. With 32 subjects split evenly, each treatment gets 32 / 4 = 8 subjects.

The response variable is the time to tie the knot, in seconds.

Age is recorded but not a factor: Abebe does not set anyone’s age, he only writes it down. A factor is something the experimenter imposes.

Example 3

Counting treatments and units

Next season Farrukh wants to add a third factor, soybean variety, at 2 levels, keeping fertilizer at 3 levels and watering at 2, with 4 plots per treatment. How many treatments and plots does that take? His field has only 30 plots; how many plots per treatment can he give if every treatment gets the same number?

Solution

Every combination of one level of each factor is a treatment, so there are

2 \times 3 \times 2 = 12 \text{ treatments},

and with 4 plots each, 12 \times 4 = 48 plots.

With 30 plots, 30 / 12 = 2.5, which is not a whole number. Equal numbers per treatment means 2 plots each, using 12 \times 2 = 24 of the 30 plots. Adding a factor multiplies the number of treatments, so it multiplies the number of units needed too.

Example 4

Treatment means from plot yields

At harvest the 24 plots of Figure 1 give these yields, in bushels per acre (made-up data). Find the mean yield for each treatment.

	fertilizer	water	yields
1	none	light	36, 39, 37, 40
2	40	light	47, 50, 46, 49
3	80	light	45, 42, 46, 43
4	none	heavy	40, 43, 39, 42
5	40	heavy	51, 48, 52, 49
6	80	heavy	57, 60, 56, 59

Solution

A treatment’s mean uses its own 4 plots and nothing else. For treatment 1,

\bar{y} = \frac{36 + 39 + 37 + 40}{4} = \frac{152}{4} = 38.

The others work the same way: treatment 2, 192 / 4 = 48; treatment 3, 176 / 4 = 44; treatment 4, 164 / 4 = 41; treatment 5, 200 / 4 = 50; treatment 6, 232 / 4 = 58 bushels per acre. Arranged in the grid of Figure 1:

	none	40	80
light	38	48	44
heavy	41	50	58

These six means are the points of Figure 2.

Example 5

Measuring the interaction

Use the treatment means of Example 4. Find the effect of raising fertilizer from 40 to 80 pounds per acre at each watering level, and the effect of heavy rather than light watering at each fertilizer level. Do the factors interact?

Solution

An effect is a difference in mean yield between two treatments that differ in one factor only.

Raising fertilizer from 40 to 80:

\begin{aligned} \text{light: } & 44 - 48 = -4, \\ \text{heavy: } & 58 - 50 = 8. \end{aligned}

With light watering the extra fertilizer costs 4 bushels per acre; with heavy watering it adds 8. Heavy rather than light watering:

\begin{aligned} \text{none: } & 41 - 38 = 3, \\ 40\text{: } & 50 - 48 = 2, \\ 80\text{: } & 58 - 44 = 14. \end{aligned}

The watering effect is small at the two lower fertilizer levels and large at 80 pounds. Each factor’s effect depends on the level of the other, so the factors interact. In Figure 2 this is the gap between the lines staying about the same and then opening wide.

Example 6

What two one-factor experiments would have missed

Suppose that instead of the full trial, Farrukh had tested fertilizer with light watering only, and Ismoil had tested watering with no fertilizer only, and each found the same means as the matching treatments of Example 4. What would they have recommended, and how does it compare with the best treatment in the full trial? What would averaging over watering have shown?

Solution

Farrukh’s experiment is the light row: 38, 48, 44. He would pick 40 pounds and report that 80 pounds lowers yield. Ismoil’s experiment is the “none” column: 38 light, 41 heavy. He would pick heavy watering. Put together, they recommend 40 pounds with heavy watering, treatment 5, with a mean of 50.

The full trial’s best treatment is 80 pounds with heavy watering, with a mean of 58, which is

58 - 50 = 8 \text{ bushels per acre}

more. Neither one-factor experiment grew that treatment, and Farrukh’s report that 80 pounds lowers yield is true only with light watering.

Averaging over watering does not fix this. The mean of the 40-pound treatments is (48 + 50)/2 = 49, and of the 80-pound treatments (44 + 58)/2 = 51: a gain of 2 that describes neither row, where the actual effects were -4 and 8. When factors interact, the effect of one has to be reported at each level of the other.

Boundaries

The unit is what gets assigned, not what gets measured. If a school gives a schedule to whole classrooms, the classroom is the unit, even if each student takes a test. Counting the students as units would pretend there were more independent assignments than there were.

A factor must be set by the experimenter. A variable that is only recorded (a subject’s age, a plot’s soil type measured but not changed) is not a factor, and comparing its groups is observational, with the limits of Correlation is not causation: lurking and confounding variables.

The count assumes every combination is used. The counting rule assumes all k_1 \times k_2 \times \cdots combinations are run. A design that skips some cannot measure the interaction at the skipped boxes.

Levels are the values tried, nothing more. The trial says nothing directly about 60 or 120 pounds per acre, or about watering heavier than “heavy”.

An interaction in sample means can be chance. Four plots per treatment leaves room for luck. Lines that are nearly parallel may show no real interaction, and judging whether a gap is larger than chance needs the methods of Comparing two treatments by re-randomizing and later work.

More factors cost units fast. Each added factor multiplies the treatments, which is why the number of factors studied at once is usually small.

Where this goes next

The parts named here are the raw material of Experimental design. Control groups, placebos and blinding adds a treatment that gives a baseline. Random assignment and confounding decides which units get which treatment, and Completely randomized design and replication does so with several units per treatment. Blocking: randomized block and matched pairs designs groups similar units first, for instance plots on the wet end of the field, before treatments are assigned. Comparing two treatment groups’ responses fairly is the work of Comparing two treatments by re-randomizing.

Exercises

Naming the parts

For Exercises 1 to 5, name the experimental units (or subjects), the factors and their levels, the treatments and the response variable.

A grower puts straw mulch, plastic mulch or no mulch on 18 pumpkin vines, 6 vines each, and weighs the largest pumpkin on each vine.
Forty volunteers read the same article, 20 on paper and 20 on a screen, and then answer 10 questions about it; the number answered correctly is recorded.
A tester paints wooden boards with brand A or brand B house paint and dries them at 50, 70 or 90 degrees Fahrenheit, recording the hours until each board is dry to the touch.
Twenty-four phones of one model are set to low, medium or high screen brightness, with wifi on or off, 4 phones per combination, and run until the battery is empty; the hours are recorded.
A school assigns 6 of its 12 homerooms to start each day with 10 minutes of silent reading and the other 6 to start as usual. At the end of the term every student takes a reading test, and each homeroom’s mean score is recorded.

Counting

An experiment has one factor with 4 levels and another with 3, and every treatment gets 5 units. How many treatments and units are there?
A soybean trial uses variety (2 levels), seed coating (coated or uncoated) and planting date (early, middle or late), with 2 plots per treatment. How many treatments and plots?
An experiment has 30 units and two factors, with 2 and 5 levels. If every treatment gets the same number of units, how many does each get?
Abebe plans factors with 2, 3 and 4 levels and wants at least 3 subjects per treatment. What is the smallest number of subjects he needs?

Means and interaction

Ismoil grows two soybean varieties, A and B, in rows 15 or 30 inches apart, 3 plots per treatment. Yields in bushels per acre: A at 15 inches: 52, 55, 49; A at 30 inches: 46, 48, 44; B at 15 inches: 50, 47, 53; B at 30 inches: 44, 45, 43. Find the four treatment means.
Using Exercise 10, find the effect of moving from 15-inch to 30-inch rows for each variety. Do variety and row spacing interact?
In a knot experiment like Abebe’s, the treatment means are: diagram, no practice 70 seconds; diagram, practice 46; video, no practice 52; video, practice 44. Find the effect of practice for each instruction format. Do the factors interact?
Using Exercise 12, suppose a one-factor experiment had tested practice using the video only, and found the same means. What effect of practice would it report, and how far is that from the effect with the diagram?

Reasoning

A student says the soybean trial has five treatments: none, 40, 80, light and heavy. Explain what is wrong and give the correct count.
In Exercise 5, explain why the experimental unit is the homeroom and not the student.
In Abebe’s knot experiment, explain why age is not a factor, and what he would have to do to make it one, if that is possible at all.
Explain why the interaction in Figure 2 could not have been found by two one-factor experiments, even if those used 24 plots between them.
On a graph of treatment means like Figure 2, what would parallel lines say about the effect of fertilizer at the two watering levels?

Answers

Units: the 18 vines. One factor, mulch, with levels straw, plastic and none; these 3 levels are the 3 treatments. Response: mass of the largest pumpkin on each vine.
Subjects: the 40 volunteers. One factor, reading medium, with levels paper and screen; 2 treatments. Response: number of questions answered correctly out of 10.
Units: the boards. Factors: paint brand (A, B) and drying temperature (50, 70, 90 degrees). Treatments: 2 \times 3 = 6, such as brand A at 50 degrees. Response: hours until dry to the touch.
Units: the 24 phones. Factors: brightness (low, medium, high) and wifi (on, off). Treatments: 3 \times 2 = 6, such as high brightness with wifi off. Response: hours until the battery is empty.
Units: the 12 homerooms. One factor, start of day, with levels silent reading and usual start; 2 treatments. Response: each homeroom’s mean reading score.
4 \times 3 = 12 treatments and 12 \times 5 = 60 units.
2 \times 2 \times 3 = 12 treatments and 12 \times 2 = 24 plots.
2 \times 5 = 10 treatments, so 30 / 10 = 3 units each.
2 \times 3 \times 4 = 24 treatments, each with at least 3, so 24 \times 3 = 72 subjects.
A at 15: 156/3 = 52; A at 30: 138/3 = 46; B at 15: 150/3 = 50; B at 30: 132/3 = 44 bushels per acre.
A: 46 - 52 = -6; B: 44 - 50 = -6. Wider rows cost 6 bushels per acre for both varieties, so the effect does not depend on variety: no interaction (parallel lines).
Diagram: 46 - 70 = -24 seconds; video: 44 - 52 = -8 seconds. Practice saves 24 seconds with the diagram but only 8 with the video, so the factors interact.
It would report that practice saves 8 seconds; with the diagram it saves 24, so the report is off by 24 - 8 = 16 seconds for diagram learners.
Those are the five levels of the two factors. A treatment combines one level of each, so there are 3 \times 2 = 6 treatments, such as none with light watering.
The reading start is given to a whole homeroom at once; no student in a room can get a different treatment from the rest. The unit is the smallest thing that receives its own treatment, which is the homeroom.
Abebe does not set anyone’s age; he only records it. A factor is imposed by the experimenter, and a person’s age cannot be imposed, so age can only be studied observationally.
A one-factor experiment holds the other factor at one level, so it sees one row or one column of the grid. The interaction is a difference between rows, and the treatment “80 pounds with heavy watering” lies in neither the light row nor the no-fertilizer column, so it would never be grown.
Parallel lines keep the same gap at every fertilizer level, so changing fertilizer would have the same effect under light and heavy watering: no interaction.