What randomness means
A coin is about to be tossed. Nobody can say how it will land, yet there is still a lot you can say. What exactly, and what not?
Where you are 14 in Basic probability
Why this exists
Some questions have answers you can work out before anything happens. What is 3 + 4? What day comes after a Tuesday? When you press the light switch in a room with a working bulb, does the light come on? Think carefully and you know.
Other questions cannot be settled in advance, however carefully you think. Which face of a coin lands up? Which number a die shows? Which name is pulled out of a hat? You can list what might happen, but not which of those things will.
That second kind would be hopeless to study if it had no rules at all. It does have rules, of a strange sort. One toss of a coin tells you nothing ahead of time, but a thousand tosses behave in a way you can describe quite exactly. Randomness is the name for that mix: unpredictable one at a time, regular in bulk. Everything in probability is built on it, so it is worth being exact about what it means before any probability is ever written as a number.
Definition
One performance of an action, such as one toss of a coin, is a trial. What happens on a trial, such as heads, is its outcome.
A random process (also called a chance process) is an action that can be repeated, under the same conditions each time, with these three properties.
- The possible outcomes are known in advance. Before the trial you can list everything that might happen.
- A single outcome cannot be predicted. No one can say for certain which outcome the next trial will give.
- Many trials show a steady pattern. Over a long run of trials, the share of trials that give each outcome settles down.
An outcome is random when it comes from a random process. The list in property 1 is the process’s sample space, which has a lesson of its own.
| Random process | One trial | Possible outcomes |
|---|---|---|
| Toss a coin | one toss | heads (H), tails (T) |
| Roll a die | one roll | 1, 2, 3, 4, 5, 6 |
| Spin the spinner in Figure 2 | one spin | 1, 2, 3 |
| Draw a marble from a bag of red and blue marbles without looking | one draw | red, blue |
Two more words describe how the trials behave.
A coin, die or spinner is fair when none of its outcomes has any advantage over the others, so that in the long run the shares of all the outcomes settle toward the same number. It is unfair, or biased, when some outcome is favored. Fair and random are separate properties: a die weighted so that 6 comes up more often is unfair, and it is still random, because nobody can say what the next roll will be. The lessons on equally likely outcomes and fair and unfair chance devices take this further.
Trials are independent when the outcome of one trial has no effect on the outcome of any other. Tossing a coin again and again gives independent trials: the coin does not know what it did before.
Intuition
Here are three runs of 20 tosses of a fair coin. They were made by a computer program built to imitate a fair coin tossed independently. The simulations in this lesson all come from a random-number generator started from a fixed number, called its seed, so anyone rerunning them gets exactly the same tosses.
| Run | The 20 tosses | Heads |
|---|---|---|
| 1 | HTTHT THTTT HHHTT HTTTT | 7 |
| 2 | HHTHH HHHHT TTHTH HTTTT | 11 |
| 3 | HHHHT HTHTT HHTHT HHTTH | 12 |
No two runs agree, and none of them gives exactly 10 heads. Nothing in the first nineteen tosses of a run tells you the twentieth. That is property 2 at work.
Now keep one coin going for 10{,}000 tosses and look at the totals at a few points along the way. After n tosses, write H for the number of heads and T for the number of tails. The share of heads is \displaystyle \frac{H}{n}, the number of heads divided by the number of tosses, and the gap is H - T.
| n | H | T | Share | Gap |
|---|---|---|---|---|
| 10 | 7 | 3 | 0.7 | 4 |
| 100 | 54 | 46 | 0.54 | 8 |
| 1{,}000 | 537 | 463 | 0.537 | 74 |
| 10{,}000 | 5{,}030 | 4{,}970 | 0.503 | 60 |
The share starts far from one half and closes in on it. Figure 1 draws the share after every toss, for this run and two others.
At the left edge each line leaps about, because one toss changes the share of a handful of tosses a lot. Further right, one toss hardly moves it, and all three lines flatten out close to \tfrac{1}{2}. Each run takes a different path, but all three end up near the same level. That is property 3.
Look at the last column of the table, though. The gap between the count of heads and the count of tails did not shrink: it was 4 after 10 tosses and 60 after 10{,}000. What settles is the share, not the counts.
A tossed coin obeys the laws of physics like anything else: how fast it spins, how high it goes and how it lands decide the face. What makes a hand toss random is that the result depends on these so delicately that nobody can control or foresee it. Calling a process random is a statement about what can be predicted, not a claim that nothing decides the outcome.
Results
Long-run regularity. Repeat a random process in independent trials under the same conditions. Then for each outcome there is one fixed number that the share of trials giving that outcome settles toward. Choose any margin, however small, such as 0.01. The more trials there are, the more likely it is that the share lands within that margin of the fixed number, and with enough trials it becomes as close to certain as you like. For a fair coin the number is \tfrac{1}{2}, and for each face of a fair die it is \tfrac{1}{6}.
This is a statement about likelihood in the long run only. It is never a certainty: a fair coin can land heads on every toss, however many there are; such runs just become very rare. And it promises nothing about any particular number of trials: after n tosses of a fair coin, for any n, the share of heads may be some way from \tfrac{1}{2}, and no number of tosses guarantees it is exactly \tfrac{1}{2} or that the counts of heads and tails even out. For instance, in 10{,}000 simulated runs of 10 tosses each, exactly 5 heads came up in only 2{,}487 runs, fewer than a quarter of them.
The share and the gap. In n tosses with H heads and T tails,
So the share of heads is close to \tfrac{1}{2} exactly when the gap H - T is small compared with 2n. The gap itself can grow while the share settles, as long as n grows faster.
No memory. If the trials are independent, what happened before has no effect on the next trial, including a long streak of one outcome. For a fair coin tossed independently, the toss after three heads in a row is still equally likely to be heads or tails. In a simulated run of 100{,}000 tosses, 12{,}543 tosses came straight after three heads in a row, and 6{,}297 of them were heads, a share of 0.502.
Why it is true
The share and the gap. Every toss is heads or tails, so H + T = n and T = n - H. Then
which is the identity. With the 10{,}000 tosses in the table, \displaystyle \frac{60}{20{,}000} = 0.003, and indeed 0.503 - 0.5 = 0.003.
Long-run regularity. The full statement is a theorem, the law of large numbers, and its proof uses tools from later courses. What can be seen now is why it is believable, by counting.
Toss a fair coin n times independently and write down the list of H’s and T’s. There are {2 \times 2 \times \cdots \times 2 = 2^n} possible lists, and for a fair coin with independent tosses every list is exactly as likely as every other. So the question “how often is the share of heads near one half?” becomes “what share of all the lists have close to half heads?” Counting the lists whose share of heads is from 0.4 to 0.6, ends included (a computer does the counting for large n):
| n | Lists, 2^n | Share of lists with 0.4n to 0.6n heads |
|---|---|---|
| 10 | 1{,}024 | \displaystyle \frac{672}{1{,}024} \approx 0.656 |
| 100 | about {1.27 \times 10^{30}} | about 0.965 |
| 1{,}000 | about {1.07 \times 10^{301}} | all but about 1.8 in every 10 billion |
For 10 tosses, about a third of all lists have a share outside 0.4 to 0.6. For 1{,}000 tosses almost none do. Lists far from half heads still exist, and each is as possible as any other list, but there are so few of them compared with the rest that a long run almost never produces one. The same happens with any margin you choose in place of 0.1: however small it is, the share of lists within that margin of one half climbs toward 1 as n grows, though a smaller margin needs a larger n. That crowding toward the middle is what makes the share settle.
Worked examples
Random or not?
For each action, decide whether it is a random process.
- (a) Toss a coin and note which face lands up.
- (b) Press the switch in a room with a working bulb and note whether the light comes on.
- (c) Shuffle a deck of cards well and note the suit of the top card.
- (d) Note the date of the day after 14 March.
- (e) Toss a coin with heads on both sides and note which face lands up.
Test each against the three properties of the definition.
(a) Random. The outcomes, heads and tails, are known in advance; no one can say which the next toss gives; and over many tosses the share of heads settles.
(b) Not random. The outcome is known in advance: with a working bulb and switch the light comes on. Property 2 fails.
(c) Random. The four suits are the possible outcomes, a good shuffle leaves nobody able to say which suit is on top, and repeating the shuffle many times gives a settled share for each suit.
(d) Not random. It is always 15 March. There is only one possible outcome and it is known.
(e) Not random. The list of outcomes has only heads on it, so the outcome is known before the toss. It is still a toss, but a toss is not what makes a process random; being unpredictable is.
What can be said before rolling a die
A fair die is about to be rolled. For each statement, say whether it must be true, cannot be true, or cannot be known in advance.
- (a) The next roll is a whole number from 1 to 6.
- (b) The next roll is a 4.
- (c) The next roll is a 7.
- (d) In 600 rolls, each face comes up exactly 100 times.
(a) Must be true. The possible outcomes are known in advance, and every one of them is a whole number from 1 to 6.
(b) Cannot be known in advance. 4 is one of the possible outcomes, but a single outcome of a random process cannot be predicted.
(c) Cannot be true. 7 is not on the list of possible outcomes. Unpredictable does not mean that anything at all can happen.
(d) Cannot be known in advance. The long-run pattern is about shares settling, not about exact counts. Here is what 600 rolls of a simulated fair die actually gave:
| Face | Rolls | Face | Rolls |
|---|---|---|---|
| 1 | 88 | 4 | 108 |
| 2 | 118 | 5 | 82 |
| 3 | 108 | 6 | 96 |
The counts add to 88 + 118 + 108 + 108 + 82 + 96 = 600, as they must, and not one of them is 100. Every face came up somewhere between 82 and 118 times.
A spinner that is random but not fair
The spinner in Figure 2 is spun by flicking the arrow hard. Is it a random process? Is it fair?
It is random. The possible outcomes are 1, 2 and 3; a hard flick leaves nobody able to say where the arrow will stop; and the shares settle over many spins.
It is not fair. Region 1 is half of the circle while regions 2 and 3 are a quarter each, so the arrow has more room to stop in region 1. Region 1 is favored. A simulation of 1{,}000 spins agrees:
| Region | 1 | 2 | 3 |
|---|---|---|---|
| Spins | 499 | 251 | 250 |
Region 1 came up about twice as often as each of the others, which matches its being twice as large.
To make a fair spinner with three outcomes, give every region the same angle at the center. A full turn is 360^\circ, so each region needs 360^\circ \div 3 = 120^\circ.
Reading a long run
Use the table of 10{,}000 tosses in the Intuition section. Find how far the share of heads is from \tfrac{1}{2} after 1{,}000 tosses, and check the answer with the gap.
SolutionAfter 1{,}000 tosses there were 537 heads, so the share is \displaystyle \frac{537}{1{,}000} = 0.537, and
The gap is 537 - 463 = 74. The identity for the share and the gap says the distance from one half is the gap divided by 2n:
The two agree. Notice that after 10{,}000 tosses the gap was a little smaller, 60, but it is now divided by 20{,}000, which is why the share was much closer to one half, 0.503.
Choosing a device to imitate a process
A simulation uses one random process to imitate another that has the same possible outcomes and the same long-run pattern. Choose a fair coin, a fair die or a spinner to imitate each process.
- (a) Guessing the answer to a true-or-false question, with each answer equally likely.
- (b) Picking one of six classmates’ names from a hat, well mixed.
- (c) Choosing one of three lunch options, each equally likely.
- (d) A process with two outcomes, A and B, where A should come up twice as often as B in the long run.
(a) A fair coin: two outcomes, neither favored. Let heads stand for “true”.
(b) A fair die: six outcomes, none favored. Number the names 1 to 6.
(c) One toss of a coin has only two outcomes, so it cannot be split into three equal groups. A fair die can: let 1 or 2 stand for the first option, 3 or 4 for the second, and 5 or 6 for the third. Each option then gets 6 \div 3 = 2 faces, so none is favored. A spinner with three regions of 120^\circ also works.
(d) A spinner whose region for A is twice as large as its region for B. If B gets x degrees, A gets 2x, and together they fill the circle:
So B gets 120^\circ and A gets 240^\circ. This spinner is deliberately unfair, because the process it imitates is.
Boundaries
Random describes the process, not one outcome or one list. A fair coin tossed ten times can give HHHHHHHHHH, and it can give HTHTHTHTHT. With a fair coin and independent tosses, each of those lists is exactly as likely as any other list of ten. You cannot look at one list and decide it is “random” or “not random”; you ask how it was made.
Unpredictable does not mean anything can happen. The outcome always comes from the known list. A die never shows 7.
Random does not mean fair. A random process may favor some outcomes, as the spinner in Figure 2 does. Fairness is an extra condition.
The long-run pattern needs the same conditions and independent trials. If the conditions change partway through (a coin gets bent, the spinner is swapped for another), the shares need not settle toward anything in particular.
The long run says nothing about the next trial or a short run. A settled share of one half does not make the next toss more likely to be heads after a run of tails, and it does not make ten tosses give five heads.
Computer randomness is made by a rule. The simulations here come from a fixed rule applied to a seed, so they are exactly repeatable, and in that sense not random at all. Using them rests on an assumption: that their output behaves like independent tosses of a fair coin. Good generators pass many tests of that kind, such as checks on shares and streaks, but no test can prove it, so it stays an assumption of the model rather than a fact about the tosses.
Some things are modeled as random without fitting the definition exactly. Tomorrow’s weather happens only once, so it cannot be repeated under the same conditions. It is still often treated as random, because that is the most useful way to reason about something nobody can predict.
Where this goes next
The next step is to put words and then numbers on chance: first describing chance in words, then the probability scale from zero to one. Listing every outcome carefully is the sample space, and when all the outcomes are equally likely the chance of an event becomes a fraction you can count, the probability of an event.
The steady long-run share from property 3 is the other road to the same number. The share you see in real trials is the relative frequency; experimental and theoretical probability compares it with the counted fraction, and the law of large numbers is the exact form of the settling you saw in Figure 1. Imitating one random process with another is developed properly in simulation, and the “no memory” property is made precise in independent events.
Randomness is also a tool on purpose. In statistics, random sampling and random assignment use a chance process to choose people, exactly because nobody can then steer the choice.
Exercises
Is it random?
Decide whether each action is a random process. Give the reason.
- Roll a die and note the number on top.
- Type 5 + 5 into a working calculator and note the answer shown.
- Write the names of the 25 students in a class on identical slips, mix them well in a hat, and draw one without looking.
- Toss a coin that has tails on both sides and note which face lands up.
- Note whether it rains in your town tomorrow.
What can be said in advance
In Exercises 6 to 9 a fair die is rolled. Say whether each statement must be true, cannot be true, or cannot be known in advance.
- The roll is an odd number.
- The roll is less than 7.
- The roll is 0.
- In 60 rolls, a 6 comes up exactly 10 times.
- The letters of the word BANANA are written on six identical cards, one letter on each, and one card is drawn without looking. How many different outcomes are possible if the outcome is the letter drawn? Is each letter equally likely?
Fair or unfair
- A spinner is split into four regions, each with an angle of 90^\circ at the center. Is it fair? Explain.
- A spinner is to be split into 5 regions so that it is fair. What angle should each region have at the center?
- A spinner has three regions with angles 180^\circ, 120^\circ and 60^\circ. Check that the angles fill the circle, and say which region is favored and which turns up least in the long run. Is the spinner a random process?
- A die has faces showing 1, 1, 2, 3, 4, 5. How many different outcomes are possible? Is the die fair for those outcomes? Is it a random process?
- In the 600 rolls of Example 2, face 2 came up 118 times and face 5 came up 82 times. How far is each count from 100? The die was a simulated fair die. Explain how a fair die can still give counts like these.
Reading a long run
- Use the table of 10{,}000 tosses. What was the share of heads after 100 tosses, and what was the gap between heads and tails?
- After 10{,}000 tosses the gap was 60. Use the identity for the share and the gap to find the share of heads.
- A coin is tossed 400 times, and there are 20 more heads than tails. How many heads were there, and what is the share of heads?
- A coin is tossed 2{,}000 times, and the share of heads is 0.51. How many heads and how many tails were there, and what is the gap?
Simulating
- Explain how to use a fair die to imitate a process with two equally likely outcomes.
- Design a spinner with two regions, A and B, so that A comes up three times as often as B in the long run. What angle does each region need?
- Can one toss of a fair coin imitate a choice among four equally likely options? Can two tosses? Explain.
Reasoning
- A fair coin is tossed 10 times and gives 10 heads. The tosses are independent. Is the process still random? What can be said about the eleventh toss?
- Explain, using the identity for the share and the gap, how the share of heads can get closer to \tfrac{1}{2} while the gap between heads and tails gets bigger.
- Someone hands you a list of 20 H’s and T’s. Explain why the list alone cannot tell you whether it came from tossing a fair coin.
Answers
- Random. The outcomes 1 to 6 are known in advance, the next roll cannot be predicted, and the shares settle over many rolls.
- Not random. The answer is always 10, known before you press the keys.
- Random. The 25 names are the possible outcomes, good mixing leaves nobody able to say which comes out, and the draw can be repeated (putting the slip back) with shares that settle.
- Not random. Tails is the only possible outcome, so it is known in advance.
- It does not fit the definition exactly: tomorrow happens once, so it cannot be repeated under the same conditions. It is often modeled as random all the same, because nobody can predict it for certain; that is what a forecast’s “chance of rain” does.
- Cannot be known in advance. The odd numbers 1, 3 and 5 are possible, and so are the even ones.
- Must be true. Every possible outcome, 1 to 6, is less than 7.
- Cannot be true. 0 is not a possible outcome.
- Cannot be known in advance. Exactly 10 is possible, but the long-run pattern is about shares, not exact counts.
- 3 outcomes: B, A and N. They are not equally likely: A is on 3 cards, N on 2 and B on 1, so A is favored.
- Yes, provided the spin is hard enough that nobody can control it. The four regions are the same size, 4 \times 90^\circ = 360^\circ, so no outcome has an advantage.
- 72^\circ. Equal regions share the full turn: 360^\circ \div 5 = 72^\circ.
- 180^\circ + 120^\circ + 60^\circ = 360^\circ, so they fill the circle. The 180^\circ region is favored and the 60^\circ region turns up least. It is still a random process: the next spin cannot be predicted.
- 5 outcomes: 1, 2, 3, 4, 5. Not fair, because 1 is on two faces and so is favored. It is still random.
- Each is 18 away: 118 - 100 = 18 and 100 - 82 = 18. A fair die makes the share of each face settle toward \tfrac{1}{6} only in the long run; it promises nothing about exact counts in 600 rolls, and counts some way from 100 are what a fair die gives.
- Share 0.54 and gap 8. There were 54 heads and 46 tails, so the share is \displaystyle \frac{54}{100} = 0.54 and the gap is 54 - 46 = 8.
- 0.503. \displaystyle \frac{60}{2 \times 10{,}000} = 0.003, so the share is 0.5 + 0.003 = 0.503.
- 210 heads, share 0.525. H + T = 400 and H - T = 20; adding, 2H = 420, so H = 210 and T = 190. The share is \displaystyle \frac{210}{400} = 0.525.
- 1{,}020 heads, 980 tails, gap 40. Heads are 0.51 \times 2{,}000 = 1{,}020, tails are 2{,}000 - 1{,}020 = 980, and 1{,}020 - 980 = 40.
- Split the six faces into two groups of three, for example odd (1, 3, 5) for the first outcome and even (2, 4, 6) for the second. Each group has three of the six faces, so for a fair die neither outcome is favored.
- B gets 90^\circ and A gets 270^\circ. If B gets x degrees, A gets 3x, and x + 3x = 360 gives x = 90.
- One toss cannot: it has only two outcomes, and two outcomes cannot be shared out equally among four options. Two tosses can: they give four lists, HH, HT, TH and TT, and for a fair coin tossed independently the four are equally likely, so each list can stand for one option.
- Yes, still random: every toss still has two possible outcomes and cannot be predicted. Because the tosses are independent, the first ten have no effect on the eleventh, which is equally likely to be heads or tails.
- \displaystyle \frac{H}{n} - \frac{1}{2} = \frac{H - T}{2n}. The share’s distance from one half is the gap divided by 2n. If the gap grows but n grows much faster, the fraction still shrinks: a gap of 60 over 10{,}000 tosses puts the share only 0.003 from one half.
- A fair coin tossed independently can produce every one of the 2^{20} = 1{,}048{,}576 possible lists of 20, and each list is exactly as likely as any other. So any list you are handed could have come from a fair coin, and it could equally have been typed by hand; randomness belongs to the way a list is made, not to the list.