Statistics

Deciding and concluding from a p-value

A sample says more customers buy tea than the owner believed, with a p-value of 0.020. What does she decide, what may she then say about all her customers, and what may she not say?

You need

p-values and the significance levelStatistics Confidence level and interpreting an intervalStatistics

Unlocks

Cautions about significanceStatistics

Where you are 7 in Hypothesis testing

Statistics7

Statistical inference8

Hypothesis testing7

▴ 1 above Randomization tests Test statistic

Deciding and concluding from a p-value

Type I and Type II errors Cautions about significance ▾ Show all 7 (1 more)

When you reach for it

Ebru runs a made-up simit bakery in Istanbul. She has long believed that 25\% of her customers also buy a glass of tea with their simit. Her nephew Kaan wanted to check, so before collecting any data they wrote down the hypotheses (see Null and alternative hypotheses) and a significance level. With p the proportion of all the bakery’s customers who buy tea,

H_0\colon p = 0.25 \qquad H_a\colon p \neq 0.25 \qquad \alpha = 0.05

A random sample of 320 customers then showed 98 buying tea, and the test statistic and the p-value came out at z \approx 2.32 and 0.020. Recall from p-values and the significance level that the p-value is the chance, if H_0 were true, of a sample at least as far from 0.25 as this one, and that \alpha is the cutoff for “small enough”.

The numbers are finished; the thinking is not. Ebru now needs three things: a decision (reject H_0 or not), a conclusion in words about her customers, and a sense of what the decision does not say. Part 1 of this lesson makes the decision and writes the conclusion. Part 2 matches a two-sided test at level \alpha with a confidence interval at level 1 - \alpha, a match that for proportions is close but not exact.

The procedure

The cutoff comes first. The significance level \alpha is fixed before the data are seen, chosen from the cost of a false alarm: a smaller \alpha when acting on a wrong rejection would be expensive. The reason it must come first is the guarantee behind it. If H_0 is true, a share \alpha of all samples give a p-value at or below \alpha; a cutoff picked after seeing the p-value can be fitted to the result and guarantees nothing.

The decision is one comparison.

\text{p-value} \le \alpha \;\Longrightarrow\; \text{reject } H_0

\text{p-value} > \alpha \;\Longrightarrow\; \text{fail to reject } H_0

A p-value at or below \alpha means the data would be rare enough under H_0 that “H_0 is true and this sample was unlucky” is no longer a convincing explanation. A larger p-value means results like this one happen often enough under H_0 that chance alone explains them.

The decision is about H_0; the conclusion is about the world. A decision sentence alone (“reject H_0”) says nothing to someone who does not know what H_0 was. The conclusion restates the alternative in context, naming the parameter (a proportion, of what) and the population (all of whom), and says how strong the evidence is:

after rejecting: “there is convincing evidence that [the alternative, in words]”;
after failing to reject: “there is not convincing evidence that [the alternative, in words]”.

Both sentences are about the alternative, because the test was built to look for evidence against H_0 and in favor of H_a. The words “convincing evidence” also carry the right strength. A test never proves anything; a rare sample can come from a true H_0, which is the false alarm that \alpha limits.

Failing to reject is not accepting. A large p-value says the data are consistent with H_0. They are usually consistent with many other values of p as well, and those cannot all be true. Small samples are the plainest case: a small sample varies so much that it fails to reject even when p is well away from p_0. How often a test misses a real difference is its Type II error rate, and the chance it catches one is its power. So the conclusion after failing to reject is “not convincing evidence that p differs from p_0”, never “p = p_0”. Absence of evidence is not evidence of absence.

Tests and intervals. A confidence interval at level C is built by a method that captures the true parameter in a share C of all samples; it holds the values of p that the data do not rule out (for one proportion, see One sample interval for a proportion and sample size). A two-sided test at level \alpha and the interval at level 1 - \alpha answer one question from two sides:

\begin{aligned} &\text{reject } H_0\colon p = p_0 \text{ at level } \alpha \\ &\quad\Longleftrightarrow\; p_0 \text{ lies outside the } (1 - \alpha) \text{ interval} \end{aligned}

Both use the same critical value z^*. The test rejects when \hat{p} is more than z^* standard deviations from p_0; the interval leaves p_0 out when p_0 is more than z^* standard errors from \hat{p}. Those are the same distance, measured from opposite ends. For a proportion the rule is very nearly, not exactly, true, because the two measuring sticks differ:

\sigma_{\hat{p}} = \sqrt{\frac{p_0(1 - p_0)}{n}} \qquad SE_{\hat{p}} = \sqrt{\frac{\hat{p}(1 - \hat{p})}{n}}

The test lives in the world where H_0 is true, so it uses p_0; the interval has no null value, so it uses \hat{p}. When \hat{p} is close to p_0, or the result is far from the edge, the two agree. Only a result near the edge can fall on different sides of it.

One-sided tests. A one-sided test at level \alpha puts all of \alpha in one tail. The interval with \alpha in each tail is the one at level 1 - 2\alpha, so a one-sided test at \alpha = 0.05 matches a 90\% interval, not a 95\% one: the test rejects H_0\colon p = p_0 in favor of p > p_0 when p_0 lies below the whole 90\% interval (and in favor of p < p_0 when p_0 lies above it).

Worked examples

Examples 1 to 4 belong to Part 1: the decision, the conclusion, and what failing to reject means. Examples 5 to 7 belong to Part 2: tests and intervals.

Example 1

The tea test

Carry out the decision and conclusion for Ebru’s test: H_0\colon p = 0.25, H_a\colon p \neq 0.25, \alpha = 0.05, and a random sample of 320 customers of whom 98 buy tea.

Solution

The conditions (see Conditions for one proportion) hold: the sample is random, 320 is far fewer than 10\% of all the bakery’s customers, and the expected counts under H_0 are 320(0.25) = 80 and 320(0.75) = 240, both at least 10. The sample proportion is

\hat{p} = \frac{98}{320} \approx 0.306

The test works inside the world where H_0 is true, so the standard deviation uses p_0:

\sigma_{\hat{p}} = \sqrt{\frac{0.25 \times 0.75}{320}} \approx 0.0242

z = \frac{0.30625 - 0.25}{0.0242} \approx 2.32

The alternative is two-sided, so both tails count:

\text{p-value} = 2\,P(Z \ge 2.32) \approx 0.020

Decision. 0.020 \le 0.05, so reject H_0.

Conclusion. At the 0.05 level there is convincing evidence that the proportion of all the bakery’s customers who buy tea differs from 25\%. The sample, at about 30.6\%, points above 25\%; how far above is a question for an interval (Example 5).

Example 2

One p-value, three levels

Before the data came in, Cansu, who runs the counter, had argued for \alpha = 0.10, since a wrong rejection would only mean brewing a little more tea. Kaan had argued for \alpha = 0.01, since he wanted strong evidence before telling his aunt she was wrong. State the decision and the conclusion for the p-value of 0.020 at each of \alpha = 0.10, 0.05 and 0.01.

Solution

Each level is a separate cutoff, compared with the same p-value (Figure 1).

Figure 1The bakery’s p-value, 0.020, against three significance levels. It is at or below 0.05 and 0.10, so the test rejects at those levels, and above 0.01, so it does not reject there (made-up data).

Level	Comparison	Decision
\alpha = 0.10	0.020 \le 0.10	reject H_0
\alpha = 0.05	0.020 \le 0.05	reject H_0
\alpha = 0.01	0.020 > 0.01	fail to reject H_0

At 0.10 and 0.05: there is convincing evidence that the proportion of all the bakery’s customers who buy tea differs from 25\%. At 0.01: there is not convincing evidence that it differs from 25\%.

The results do not conflict. If H_0 were true, a result at least this far from 0.25 would happen about 2\% of the time: rare enough for a 5\% or 10\% false-alarm rate, not rare enough for 1\%. The levels ask for different strengths of evidence, which is why a report gives the p-value as well as the decision.

Example 3

A small sample that proves nothing

A month earlier Kaan had taken a quick random sample of only 50 customers, and 17 bought tea. Test H_0\colon p = 0.25 against H_a\colon p \neq 0.25 at \alpha = 0.05. Then test H_0\colon p = 0.40 against H_a\colon p \neq 0.40 with the same data, and explain what the two results together show.

Solution

The sample proportion is \displaystyle \hat{p} = \frac{17}{50} = 0.34. The expected counts under p_0 = 0.25 are 12.5 and 37.5, both at least 10. Then

\sigma_{\hat{p}} = \sqrt{\frac{0.25 \times 0.75}{50}} \approx 0.0612

z = \frac{0.34 - 0.25}{0.0612} \approx 1.47

and the p-value is 2\,P(Z \ge 1.47) \approx 0.142. Since 0.142 > 0.05, fail to reject: there is not convincing evidence that the proportion of all customers who buy tea differs from 25\%.

Now the null value 0.40. The expected counts are 20 and 30, and

\sigma_{\hat{p}} = \sqrt{\frac{0.40 \times 0.60}{50}} \approx 0.0693

z = \frac{0.34 - 0.40}{0.0693} \approx -0.87

with p-value 2\,P(Z \le -0.87) \approx 0.386. Again fail to reject.

If “fail to reject” meant “H_0 is true”, the same 50 customers would show both p = 0.25 and p = 0.40, which is impossible. A sample of 50 is simply too small to tell these values apart: its proportion varies by about 0.06 from sample to sample, and both null values sit within about 1.5 of those steps of 0.34. Failing to reject reflects little data, not a confirmed 25\%.

Example 4

A one-sided test at a strict level

Cansu suspects that more than 40\% of the bakery’s customers buy two or more simits. Because Ebru would change her morning baking on the strength of the result, they fix \alpha = 0.01 in advance, with p the proportion of all the bakery’s customers who buy two or more simits. A random sample of 200 customers has 93 who do. Make the decision and write the conclusion.

Solution

H_0\colon p = 0.40 and H_a\colon p > 0.40. The expected counts are 200(0.40) = 80 and 120, and the sample is random. Then

\hat{p} = \frac{93}{200} = 0.465

\sigma_{\hat{p}} = \sqrt{\frac{0.40 \times 0.60}{200}} \approx 0.0346

z = \frac{0.465 - 0.40}{0.0346} \approx 1.88

The alternative points up, so the p-value is the right tail, P(Z \ge 1.88) \approx 0.030.

Decision. 0.030 > 0.01, so fail to reject H_0.

Conclusion. At the 0.01 level there is not convincing evidence that more than 40\% of all the bakery’s customers buy two or more simits.

The conclusion does not say that 40\% is the true share (Example 3), and the level stays at the 0.01 agreed in advance, the only level that carries its false-alarm guarantee. Reporting the p-value, 0.030, shows that the evidence was fairly strong but short of what they asked for.

Example 5

The test and the intervals

Find the 95\% and 99\% confidence intervals for the proportion of all the bakery’s customers who buy tea, from the sample of 320 with 98 buying tea, and compare each with the test of H_0\colon p = 0.25 at the matching level.

Solution

The interval has no null value to assume, so its standard error uses \hat{p} = 0.30625:

SE_{\hat{p}} = \sqrt{\frac{0.30625 \times 0.69375}{320}} \approx 0.0258

The counts 98 and 222 are both at least 10. For 95\%, z^* = 1.96:

0.306 \pm 1.96(0.0258) \approx 0.306 \pm 0.051

which, from the unrounded \hat{p} and margin, is about 0.256 to 0.357. For 99\%, z^* = 2.576:

0.306 \pm 2.576(0.0258) \approx 0.306 \pm 0.066

which is about 0.240 to 0.373 (Figure 2).

Figure 2The 95% interval leaves out 0.25, matching the rejection at level 0.05; the 99% interval takes 0.25 in, matching the failure to reject at level 0.01 (made-up data).

The 95\% interval lies entirely above 0.25, so 0.25 is not a plausible value at that level, and the test at \alpha = 1 - 0.95 = 0.05 rejected (Example 1). The 99\% interval takes 0.25 in, and the test at \alpha = 0.01 failed to reject (Example 2). Each pair agrees.

The interval also says what the test does not: Ebru can be 95\% confident that between about 26\% and 36\% of her customers buy tea, so it says roughly how far from 25\% the share is.

Example 6

Where the match is not exact

A year later Kaan takes a fresh random sample of 300 customers and finds 90 buying tea. Run the two-sided test of H_0\colon p = 0.25 at \alpha = 0.05 and find the 95\% interval. Do they agree?

Solution

\displaystyle \hat{p} = \frac{90}{300} = 0.30, and the expected counts are 75 and 225. For the test,

\sigma_{\hat{p}} = \sqrt{\frac{0.25 \times 0.75}{300}} = 0.025

z = \frac{0.30 - 0.25}{0.025} = 2.00

and the p-value is 2\,P(Z \ge 2.00) \approx 0.0455. Since 0.0455 \le 0.05, reject H_0.

For the interval,

SE_{\hat{p}} = \sqrt{\frac{0.30 \times 0.70}{300}} \approx 0.0265

0.30 \pm 1.96(0.0265) \approx 0.30 \pm 0.052

which runs from about 0.248 to 0.352. That interval contains 0.25.

So they disagree, barely. The test rejects because 0.30 is 2.00 of its standard deviations from 0.25, past 1.96. The interval keeps 0.25 because, measured with its standard error, the distance is \displaystyle \frac{0.05}{0.0265} \approx 1.89, short of 1.96. The difference comes entirely from the two square roots: 0.30 \times 0.70 = 0.21 is larger than 0.25 \times 0.75 = 0.1875, so the interval’s measuring stick is longer. The question asked was a test of 0.25, so the test’s decision stands: convincing at the 0.05 level, but only just.

Example 7

A one-sided test against an interval

For Cansu’s data in Example 4 (93 of 200, H_a\colon p > 0.40), find the 90\%, 95\% and 98\% intervals. Which interval matches a one-sided test at \alpha = 0.05, and which matches the level 0.01 that was actually used?

Solution

SE_{\hat{p}} = \sqrt{\frac{0.465 \times 0.535}{200}} \approx 0.0353

With z^* = 1.645, 1.96 and 2.326:

Level	Margin of error	Interval
90\%	1.645(0.0353) \approx 0.058	0.407 to 0.523
95\%	1.96(0.0353) \approx 0.069	0.396 to 0.534
98\%	2.326(0.0353) \approx 0.082	0.383 to 0.547

A one-sided test at \alpha = 0.05 puts the whole 0.05 in the upper tail, so it rejects when z \ge 1.645. The interval with 0.05 in each tail is the 90\% interval, 1 - 2(0.05) = 0.90. Had the level been 0.05, the p-value 0.030 would reject, and 0.40 lies below the whole 90\% interval: they agree. The 95\% interval contains 0.40, but it matches a two-sided test at 0.05, a different question.

At the level actually used, \alpha = 0.01, the matching interval is at 1 - 2(0.01) = 0.98. It contains 0.40, in line with the failure to reject in Example 4.

Boundaries

The decision is only as good as the p-value. A p-value from a sample that was not random, or from a model whose conditions fail, does not measure chance variation, and comparing it with \alpha decides nothing.

\alpha and the alternative are fixed before the data. Choosing the level, or the direction of a one-sided test, after looking at the data voids the false-alarm guarantee.

“Fail to reject” never means “accept”. The data are consistent with H_0 and with other values near it. Only an interval says which values those are.

Rejecting does not say how big the difference is. With a large sample a tiny difference can give a small p-value; whether it matters is a separate question, taken up in Cautions about significance.

The interval match is exact only when both use the same standard error. For one proportion the test uses p_0 and the interval uses \hat{p}, so near the edge they can disagree (Example 6). For a one-sample test of a mean with the t distribution, both use \displaystyle s/\sqrt{n} and the match is exact.

The match needs the right pair. A two-sided test at level \alpha goes with the 1 - \alpha interval; a one-sided test at level \alpha goes with the 1 - 2\alpha interval, and only when \hat{p} falls on the side the alternative names.

Where this goes next

The decision and conclusion close every significance test. The one-sample z-test for a proportion puts hypotheses, conditions, the test statistic, the p-value and this lesson’s conclusion into one complete procedure, and Hypothesis testing repeats it for means, differences and slopes. Each decision can be wrong in one of two ways, studied in Type I and Type II errors, and how likely a test is to catch a real effect is its power. Cautions about significance asks whether a rejected H_0 means a difference worth caring about.

Exercises

Part 1: deciding at a level set in advance

State the decision for each: (a) p-value 0.034, \alpha = 0.05; (b) p-value 0.034, \alpha = 0.01; (c) p-value 0.087, \alpha = 0.10; (d) p-value 0.087, \alpha = 0.05.
A two-sided test gives a p-value of 0.003. State the decision at \alpha = 0.10, 0.05 and 0.01.
A test gives a p-value of 0.062. State the decision at \alpha = 0.10, 0.05 and 0.01.
Ebru believes 70\% of her customers pay by card. Kaan tests whether that share has changed, in either direction, at \alpha = 0.05 set in advance. In a random sample of 250 customers, 161 pay by card. State the hypotheses, find z and the p-value, and give the decision and the conclusion in context.
Cansu suspects that fewer than half of the bakery’s customers ask for a paper bag. At \alpha = 0.05, a random sample of 180 customers has 76 who ask. State the hypotheses, find z and the p-value, and give the decision and conclusion.
Ebru suspects that more than 15\% of customers also buy a pastry. She fixes \alpha = 0.10 in advance. In a random sample of 120 customers, 24 buy one. Find z and the p-value, give the decision and conclusion, and say what the decision would have been at \alpha = 0.05.
A test of H_0\colon p = 0.30 against H_a\colon p \neq 0.30, where p is the proportion of all the bakery’s weekday customers who pay in cash, gives a p-value of 0.41 at \alpha = 0.05. Write the decision and the conclusion in context.
For the test in Exercise 7, say whether each sentence states the result correctly: (a) “We accept that 30\% of weekday customers pay in cash.” (b) “There is not convincing evidence that the proportion of all weekday customers who pay in cash differs from 30\%.” (c) “The test proves the share is 30\%.” (d) “The data are consistent with a cash share of 30\%.”
In a random sample of 60 customers, 21 buy tea. At \alpha = 0.05, test H_0\colon p = 0.25 against H_a\colon p \neq 0.25, and then H_0\colon p = 0.45 against H_a\colon p \neq 0.45. Give both p-values and explain why neither result shows its null hypothesis to be true.

Part 2: tests and intervals

A 95\% interval for the proportion of all the bakery’s customers who sit down to eat inside is 0.412 to 0.508. Using it, decide the two-sided test at \alpha = 0.05 for each null value: (a) p_0 = 0.50; (b) p_0 = 0.40; (c) p_0 = 0.45.
A 99\% interval for the proportion of customers who arrive before 8 a.m. is 0.18 to 0.27. (a) Decide the two-sided test of p_0 = 0.30 at \alpha = 0.01. (b) Decide it for p_0 = 0.20. (c) Without new work, decide the two-sided test of p_0 = 0.30 at \alpha = 0.05, and explain.
For Kaan’s card data in Exercise 4 (161 of 250), find the 95\% interval for the proportion who pay by card. Does it agree with the test?
Cansu’s test in Exercise 5 is one-sided at \alpha = 0.05. Which confidence level matches it? Find that interval from her data (76 of 180) and say whether it agrees with the test.
In winter Kaan takes a random sample of 250 customers, and 50 buy tea. For all winter customers, test H_0\colon p = 0.25 against H_a\colon p \neq 0.25 at \alpha = 0.05, and find the 95\% interval. Do they agree? Explain.
Give the confidence level that matches each test: (a) two-sided, \alpha = 0.10; (b) two-sided, \alpha = 0.01; (c) one-sided, \alpha = 0.05; (d) one-sided, \alpha = 0.01.
A 90\% interval for the proportion of customers who buy a simit with cheese is 0.214 to 0.286. What does it say about the two-sided test of H_0\colon p = 0.25 at \alpha = 0.10, and may Ebru conclude that exactly a quarter of her customers buy one?

Reasoning

Explain, using a sample that fails to reject two different null values, why “fail to reject H_0” is not the same as “accept H_0”.
Explain why a two-sided test that rejects at \alpha = 0.01 must also reject at \alpha = 0.05, once using p-values and once using intervals.
Explain why, for a proportion, a two-sided test and the matching interval can disagree, and why they can only do so for results near the edge.
Suppose Kaan’s sample in Example 3 had been 500 customers with 170 buying tea, the same \hat{p} = 0.34. Find z and the p-value for H_0\colon p = 0.25, and explain why the decision changes.
A one-sided test at \alpha = 0.05 rejects H_0\colon p = 0.40 in favor of p > 0.40, yet the 95\% interval contains 0.40. Is that a contradiction? Explain.
Explain why a conclusion must name the parameter and the population, and not only say “reject H_0”.

Answers

(a) Reject, 0.034 \le 0.05. (b) Fail to reject, 0.034 > 0.01. (c) Reject, 0.087 \le 0.10. (d) Fail to reject, 0.087 > 0.05.
Reject at all three levels, since 0.003 is at or below 0.10, 0.05 and 0.01.
Reject at 0.10 (0.062 \le 0.10); fail to reject at 0.05 and at 0.01.
H_0\colon p = 0.70, H_a\colon p \neq 0.70, p the proportion of all the bakery’s customers who pay by card. Expected counts 175 and 75. \displaystyle \hat{p} = \frac{161}{250} = 0.644, \displaystyle \sigma_{\hat{p}} = \sqrt{\frac{0.70 \times 0.30}{250}} \approx 0.0290, z \approx -1.93, p-value = 2\,P(Z \le -1.93) \approx 0.053. Since 0.053 > 0.05, fail to reject: there is not convincing evidence that the proportion of all customers who pay by card differs from 70\%.
H_0\colon p = 0.50, H_a\colon p < 0.50, p the proportion of all customers who ask for a bag. Expected counts 90 and 90. \displaystyle \hat{p} = \frac{76}{180} \approx 0.422, \displaystyle \sigma_{\hat{p}} = \sqrt{\frac{0.50 \times 0.50}{180}} \approx 0.0373, z \approx -2.09, p-value = P(Z \le -2.09) \approx 0.018. Reject: there is convincing evidence that fewer than half of all the bakery’s customers ask for a paper bag.
H_0\colon p = 0.15, H_a\colon p > 0.15. Expected counts 18 and 102. \displaystyle \hat{p} = \frac{24}{120} = 0.20, \displaystyle \sigma_{\hat{p}} = \sqrt{\frac{0.15 \times 0.85}{120}} \approx 0.0326, z \approx 1.53, p-value = P(Z \ge 1.53) \approx 0.063. Since 0.063 \le 0.10, reject: there is convincing evidence that more than 15\% of all customers buy a pastry. At \alpha = 0.05 it would have been fail to reject, 0.063 > 0.05.
0.41 > 0.05, so fail to reject H_0. There is not convincing evidence that the proportion of all the bakery’s weekday customers who pay in cash differs from 30\%.
(a) Not correct: failing to reject does not accept H_0. (b) Correct. (c) Not correct: a test proves nothing, and least of all H_0. (d) Correct: a large p-value means data like these are common if the share is 30\%.
\displaystyle \hat{p} = \frac{21}{60} = 0.35. For 0.25: \sigma_{\hat{p}} \approx 0.0559, z \approx 1.79, p-value \approx 0.074. For 0.45: \sigma_{\hat{p}} \approx 0.0642, z \approx -1.56, p-value \approx 0.119. (Expected counts 15 and 45; 27 and 33.) Both fail to reject at 0.05. The share cannot be both 0.25 and 0.45, so neither result shows its null true; a sample of 60 is too small to rule either out.
(a) 0.50 is inside: fail to reject. (b) 0.40 is outside: reject. (c) 0.45 is inside: fail to reject.
(a) 0.30 is outside: reject. (b) 0.20 is inside: fail to reject. (c) Reject. The 95\% interval is narrower than the 99\% interval and sits inside it, so 0.30 is outside it too.
\displaystyle SE_{\hat{p}} = \sqrt{\frac{0.644 \times 0.356}{250}} \approx 0.0303, margin 1.96(0.0303) \approx 0.059, interval about 0.585 to 0.703. It contains 0.70, which agrees with failing to reject.
The 90\% level, 1 - 2(0.05). \displaystyle SE_{\hat{p}} = \sqrt{\frac{0.422 \times 0.578}{180}} \approx 0.0368, margin 1.645(0.0368) \approx 0.061, interval about 0.362 to 0.483. The whole interval is below 0.50, which agrees with rejecting in favor of p < 0.50.
\displaystyle \hat{p} = \frac{50}{250} = 0.20. Test: \sigma_{\hat{p}} \approx 0.0274, z \approx -1.83, p-value \approx 0.068, fail to reject. Interval: SE_{\hat{p}} \approx 0.0253, margin 1.96(0.0253) \approx 0.0496, about 0.1504 to 0.2496, which leaves out 0.25. They disagree, narrowly: 0.20 \times 0.80 = 0.16 is smaller than 0.25 \times 0.75 = 0.1875, so the interval’s standard error is shorter. The question was a test, so the decision is fail to reject, a result close to the edge.
(a) 90\%. (b) 99\%. (c) 90\%, since 1 - 2(0.05) = 0.90. (d) 98\%, since 1 - 2(0.01) = 0.98.
0.25 is inside the interval, so the test fails to reject at \alpha = 0.10. Ebru may not conclude the share is exactly a quarter: every value from 0.214 to 0.286 is plausible.
In Exercise 9 the same data fail to reject both p = 0.25 and p = 0.45. If failing to reject meant accepting, the share would be both, which is impossible. Failing to reject only says the data cannot rule that value out.
P-values: a p-value at or below 0.01 is also at or below 0.05. Intervals: the 95\% interval lies inside the 99\% one, so a value outside the 99\% interval is outside the 95\% one too.
The test measures the distance from \hat{p} to p_0 with \displaystyle \sqrt{p_0(1 - p_0)/n}, the interval with \displaystyle \sqrt{\hat{p}(1 - \hat{p})/n}. Both compare it with the same z^*, so they disagree only when \lvert \hat{p} - p_0 \rvert falls between z^* times one root and z^* times the other. The two roots differ little, so that band is narrow and sits right at the edge of rejection.
\displaystyle \sigma_{\hat{p}} = \sqrt{\frac{0.25 \times 0.75}{500}} \approx 0.0194, z \approx 4.65, p-value below 0.0001: reject. The same distance 0.09 is now many more standard deviations, because a larger sample varies less.
No. A one-sided test at 0.05 matches the 90\% interval. The 95\% interval matches a two-sided test at 0.05, which asks a different question.
“Reject H_0” means nothing without knowing what H_0 was. Naming the parameter (the proportion who do what) and the population (all of whom) says which claim about which group the data support, and how far the conclusion reaches.