Test for a difference of means
Two groups have different averages in their samples. Is the gap big enough to show that the population means really differ, or that a treatment really changed the mean, or could chance alone have produced it?
Where you are 7 in Inference for means
When you reach for it
Malee buys longan fruit from orchards in northern Thailand and pays by weight. She believes orchard A’s fruit run heavier than orchard B’s. A random sample of 25 fruit from A averages 11.8 grams and a random sample of 22 from B averages 10.9 grams (made-up data throughout this lesson). Even two orchards with equal means would give samples that differ a little. Is 0.9 grams more than chance would plausibly produce?
The two-sample t test answers that, for two populations sampled separately (Day 1) or two treatment groups in an experiment (Day 2). From the One sample test for a mean it takes the shape: a null value, a t statistic, a p-value from a t curve. From the Interval for a difference of means it takes the standard error \displaystyle \sqrt{s_1^2/n_1 + s_2^2/n_2} and the two choices of degrees of freedom. The parallel test for shares is the Test for a difference of proportions.
The procedure
Day 1: two independent random samples
Parameters and hypotheses. \mu_1 and \mu_2 are the means of a stated variable, with units, over two stated populations. The null hypothesis says there is no difference, and the alternative says what the investigator suspected before seeing the data:
The hypotheses are about the population means, never the sample means, which are already known. Naming which group is 1 fixes the order of subtraction.
Conditions, for each group. These are the Conditions for one mean, checked twice, plus independence between the samples.
- Random and independent. Each sample is a random sample from its own population, and the two are drawn independently of each other.
- Ten percent. Each sample is at most a tenth of its own population.
- Normal or large sample, each group judged on its own. A group passes if its population is close to normal, or its sample has n \ge 30 (unless the data are very strongly skewed), or a graph of its small sample shows no strong skew and no outliers. A large clean sample in one group does nothing for a small skewed sample in the other: each sample mean needs its own sampling distribution close to normal.
The test statistic. If H_0 is true, \bar{x}_1 - \bar{x}_2 varies around 0 with standard error
and the statistic counts how many standard errors the observed difference sits from 0:
Since both s_1 and s_2 estimate unknown spreads, the statistic follows a The t distribution and degrees of freedom approximately, not exactly.
Degrees of freedom. Technology uses the Welch value, given in full in the interval lesson, which is usually not a whole number. By hand, the conservative choice is the smaller of n_1 - 1 and n_2 - 1. The Welch value is never smaller, and fewer degrees of freedom mean heavier tails, so for the same t the conservative p-value is always at least as large as the technology p-value. The conservative test therefore rejects H_0 no more often than it should; it errs on the safe side.
P-value, decision, conclusion. The p-value is the tail area beyond t in the direction H_a names, both tails for \neq: `tcdf(lower, upper, df)` on a calculator, `T.DIST.RT(t, df)` or `T.DIST.2T(|t|, df)` in a spreadsheet. If it is at most \alpha, reject H_0; otherwise fail to reject (Deciding and concluding from a p-value). The conclusion is a sentence about the population means, in context, naming \alpha and the p-value.
Technology. A calculator’s 2-SampTTest takes the six summary numbers (or two lists), the alternative, and a “Pooled” choice, which should be No for the same reason as in the interval lesson. It prints t, the p-value and the Welch degrees of freedom, and checks no condition.
Day 2: randomized experiments, one side or two, and the interval
Random assignment. In an experiment, one set of subjects is split at random into two treatment groups (Random assignment and confounding). Now \mu_1 and \mu_2 are the mean responses if all the subjects got treatment 1, or all got treatment 2, and H_0 says the treatment makes no difference to the mean. The arithmetic is unchanged. The random condition is met by the random assignment, whose chance is what would make the groups differ if H_0 were true; the ten percent condition drops out, since nothing was sampled; the normal condition is checked for each group. Because random assignment makes the groups alike, apart from chance, in everything but the treatment, rejecting H_0 is evidence that the treatment caused the difference, for these subjects. They reach further only if they were themselves a random sample of a larger group.
One-sided or two-sided. The alternative is fixed before the data are seen. A two-sided p-value is twice the one-sided p-value in the direction the data fell, so a gap that is convincing one-sided may not be two-sided. Picking the side after seeing which way the data went would halve the p-value with no new evidence.
Agreement with the interval. The test and the two-sample t interval use the same standard error and, when built with the same degrees of freedom, the same t curve. A two-sided test at level \alpha rejects \mu_1 - \mu_2 = 0 exactly when |t| > t^* for confidence 1 - \alpha, which is exactly when |\bar{x}_1 - \bar{x}_2| > t^*\,\text{SE}, which is exactly when 0 lies outside the interval. A one-sided test at level \alpha matches the interval at confidence 1 - 2\alpha in the same way, with 0 needing to lie outside on the side H_a predicts.
Paired data are different. Two measurements on the same individuals, such as the same trees in two seasons, are not independent samples; they need the procedure for Paired data: interval and test for a mean difference.
Worked examples
Day 1: the two orchards, by hand
Malee’s samples are random from each orchard’s harvest of tens of thousands of fruit. Orchard A: n_A = 25, \bar{x}_A = 11.8 g, s_A = 1.6 g. Orchard B: n_B = 22, \bar{x}_B = 10.9 g, s_B = 1.9 g. Neither sample shows strong skew. Test at \alpha = 0.05 whether orchard A’s fruit are heavier on average, using the conservative degrees of freedom.
SolutionParameters. Let \mu_A be the mean mass of all fruit in orchard A’s harvest and \mu_B the same for orchard B. Malee suspected A was heavier before she weighed anything, so
Conditions. Each sample is random from its own orchard and drawn separately, so random and independent. 25 and 22 are far below a tenth of tens of thousands. Both samples are under 30, so each needs its own look at the data: neither shows strong skew or outliers. All conditions hold.
Test statistic.
P-value. Conservative \text{df} = 21, the smaller of 24 and 21. The alternative points up, so the p-value is the upper tail (Figure 1). The t table row for 21 degrees of freedom has 1.721 at tail area 0.05 and 2.080 at 0.025; since 1.721 < 1.743 < 2.080, the p-value lies between 0.025 and 0.05. Technology, `tcdf(1.7434, 10^99, 21)`, gives about 0.0479.
Decision and conclusion. Since 0.0479 \le 0.05, reject H_0. If the two orchards had equal mean fruit mass, samples like these would show orchard A ahead by 0.9 grams or more only about 4.8\% of the time. At the \alpha = 0.05 level there is convincing evidence that the mean mass of orchard A’s fruit is greater than that of orchard B’s, though the p-value is just under \alpha.
Day 1: reading 2-SampTTest output
Kittisak enters Malee’s six numbers into 2-SampTTest with alternative \mu_1 > \mu_2 and Pooled: No.
| 2-SampTTest | |
|---|---|
| alternative | \mu_1 > \mu_2 |
| t | 1.7434 |
| p | 0.04435 |
| df | 41.311 |
| \bar{x}_1, \bar{x}_2 | 11.8, 10.9 |
| s_1, s_2 | 1.6, 1.9 |
| n_1, n_2 | 25, 22 |
Confirm the degrees of freedom, explain why the p-value is smaller than in the previous example, and say whether the decision changes.
SolutionThe statistic t = 1.7434 matches the hand value; it does not depend on the degrees of freedom. With the two terms from before, s_1^2/n_1 = 0.1024 and s_2^2/n_2 \approx 0.164091, the Welch formula gives
which matches. With about 41 degrees of freedom in place of 21, the t curve has thinner tails, so the same t = 1.7434 cuts off less area: 0.0443 against 0.0479. Here both are at most 0.05, so both reject H_0 and the conclusion stands; the technology value is the more accurate.
Day 1: when the two p-values fall on either side of 0.05
Buyers prefer a small seed, since it leaves more flesh. Kittisak suspects orchard C’s fruit have lighter seeds than orchard D’s. He weighs the seeds of a random sample of 12 fruit from C, \bar{x}_C = 2.36 g, s_C = 0.26 g, with a dot plot showing no strong skew and no outliers, and a random sample of 40 fruit from D, \bar{x}_D = 2.52 g, s_D = 0.34 g, whose histogram is mildly skewed to the right. Both harvests are large. He plans to report the technology result. Test at \alpha = 0.05, with the conservative degrees of freedom and with technology’s \text{df} \approx 23.44.
SolutionHypotheses. With \mu_C, \mu_D the mean seed masses of all fruit in each harvest, H_0\colon \mu_C - \mu_D = 0 and H_a\colon \mu_C - \mu_D < 0.
Conditions, group by group. Both samples are random, independent, and small against large harvests. Orchard C’s sample has only 12 fruit, so it passes on its dot plot. Orchard D’s has 40, so mild skew is fine. The groups pass in different ways, and both pass.
Statistic.
P-values. The lower tail. Conservatively, \text{df} = 11: the table row has 1.363 at tail area 0.10 and 1.796 at 0.05, so by symmetry the p-value is between 0.05 and 0.10; technology gives `tcdf(-10^99, -1.7331, 11)` \approx 0.0555. With 23.44 degrees of freedom the p-value is about 0.0481.
Decision. The conservative test fails to reject at 0.05; the Welch test rejects. They disagree because the evidence sits right at the cutoff, and the conservative method can fail to reject when Welch rejects, never the reverse. So the decision depends on the method, and the method, like \alpha, is fixed before the data are seen. Kittisak planned on technology, the closer approximation: at the \alpha = 0.05 level there is convincing evidence, p-value about 0.048, that orchard C’s fruit have a lower mean seed mass than orchard D’s. Had he planned the hand method, he would fail to reject; switching to Welch afterward because it rejects would be choosing the answer, not testing.
Either way, 0.048 and 0.055 describe almost the same, moderate strength of evidence.
Day 2: a fruit-thinning experiment
Chanida tests fruit thinning, removing part of each young cluster so the rest grow larger. She takes 32 of her trees, assigns 16 at random to thinning and 16 to no thinning, and records each tree’s mean fruit mass at harvest. Thinned: \bar{x}_T = 12.4 g, s_T = 1.1 g. Unthinned: \bar{x}_N = 11.6 g, s_N = 1.3 g. Dot plots of both groups show no strong skew and no outliers. She expected thinning to help. Test at \alpha = 0.05 with the conservative degrees of freedom.
SolutionParameters. \mu_T is the mean fruit mass per tree if all 32 trees were thinned, \mu_N if none were. H_0\colon \mu_T - \mu_N = 0, H_a\colon \mu_T - \mu_N > 0.
Conditions. Treatments were assigned at random. No population was sampled, so there is no ten percent check. Each group has 16 trees and a clean dot plot.
Statistic.
P-value. Conservative \text{df} = 15; the upper tail beyond 1.879 is about 0.0399. (Technology uses \text{df} \approx 29.20 and gives about 0.0351.)
Conclusion. Since 0.0399 \le 0.05, reject H_0. If thinning made no difference to these trees, random assignment alone would put the thinned group 0.8 grams or more ahead only about 4\% of the time. Because thinning was assigned at random, this is convincing evidence that thinning causes heavier fruit on Chanida’s 32 trees. They are her own trees, not a random sample, so the result reaches other orchards only as far as their trees resemble hers.
Day 2: two-sided, and the intervals that agree
Kittisak asks what Chanida’s data would show if she had had no expected direction. Find the two-sided p-value, and check both decisions against confidence intervals for \mu_T - \mu_N with 15 degrees of freedom (t^* = 2.131 for 95\%, t^* = 1.753 for 90\%).
SolutionThe two-sided p-value doubles the tail: 2 \times 0.0399 \approx 0.0798, above 0.05, so a two-sided test fails to reject. Same data, different question: “is there any difference?” needs more evidence than “is thinning better?”, because it splits \alpha between both tails.
The 95\% interval, -0.107 to 1.707 grams, contains 0: it agrees with the two-sided test failing to reject. The 90\% interval, 0.054 to 1.546 grams, lies wholly above 0: it agrees with the one-sided test rejecting in favor of \mu_T > \mu_N. The interval also gives the size: thinning plausibly raises mean fruit mass by about 0.05 to 1.55 grams, and whether that is worth the labor is a separate question (Cautions about significance).
Day 2: a test from raw data that fails to reject
Malee tries a calcium spray on 16 trees, 8 chosen at random to be sprayed. Mean fruit mass per tree, in grams:
| spray | 11.2, 12.0, 11.5, 12.6, 11.8, 12.3, 11.0, 12.4 |
| no spray | 11.4, 11.9, 10.8, 12.1, 11.3, 11.6, 10.9, 12.0 |
She had no expected direction. Test at \alpha = 0.05.
SolutionHypotheses. \mu_S and \mu_C are the mean fruit mass per tree if all 16 trees were sprayed, or none. H_0\colon \mu_S - \mu_C = 0, H_a\colon \mu_S - \mu_C \neq 0.
Conditions. Random assignment. Each group has only 8 trees, so each needs its graph: Figure 2 shows no strong skew and no outliers in either.
Summaries. The sprayed trees total 94.8, so \bar{x}_S = 11.85 g, and s_S \approx 0.5806 g. The unsprayed total 92.0, so \bar{x}_C = 11.5 g, and s_C \approx 0.4899 g.
Statistic.
P-value. Two tails. With conservative \text{df} = 7, 2 \times `tcdf(1.3031, 10^99, 7)` \approx 0.234; with technology’s \text{df} \approx 13.61, about 0.214.
Conclusion. Either way the p-value is far above 0.05: fail to reject H_0. These 16 trees give no convincing evidence that the calcium spray changes mean fruit mass. Nor do they show it does nothing: with 8 trees a group the standard error is large enough to hide a real effect of a few tenths of a gram.
Boundaries
- The two samples must be independent of each other. Measurements on the same individuals or on matched pairs move together, and then the sum-of-variances standard error is wrong (Paired data: interval and test for a mean difference).
- Each group needs randomness, from random sampling or random assignment, and each must pass the normal or large-sample condition on its own.
- The ten percent condition applies to each sample separately, and not at all to an experiment with no sampling.
- The t curve is an approximation even when every condition holds; the Welch degrees of freedom give the closer one, the conservative value a larger p-value.
- Rejecting H_0 shows cause only after random assignment, and only for the subjects unless they were randomly sampled. Two sampled populations can show a difference, never what caused it.
- Failing to reject is never evidence that the means are equal, and rejecting says nothing about whether the difference is large enough to matter.
Where this goes next
When both columns are measured on the same individuals, the work moves to Paired data: interval and test for a mean difference: the pairs become one column of differences for the one-sample t procedures. Whether a real effect is large enough to matter is Cautions about significance.
Exercises
All data are made up. Use the conservative degrees of freedom and find p-values with technology unless told otherwise. Unless stated, samples are independent random samples from harvests of tens of thousands of fruit, and the normal condition holds for each group.
Day 1: two independent random samples
- Peel thickness: 30 fruit from orchard A have \bar{x} = 0.62 mm and s = 0.10 mm; 28 fruit from orchard B have \bar{x} = 0.57 mm and s = 0.12 mm. Test whether the mean peel thickness differs, at \alpha = 0.05.
- Peel mass: 18 fruit from orchard A have \bar{x} = 2.35 g and s = 0.30 g; 20 from orchard B have \bar{x} = 2.50 g and s = 0.28 g. Malee suspected orchard A’s peels were lighter. Test at \alpha = 0.05.
- Chanida writes the hypotheses for Example 1 as H_0\colon \bar{x}_A = \bar{x}_B and H_a\colon \bar{x}_A > \bar{x}_B. Explain what is wrong and write them correctly.
- For each plan, say whether the two-sample t test may be used, and if not, which condition fails. (a) Samples of 10 and 35 fruit; the dot plot of the 10 is strongly skewed to the right. (b) Fruit picked from the lowest branches, the easiest to reach. (c) A random sample of 50 fruit from a small trial plot that yielded 400 fruit, against 50 from a large harvest. (d) Samples of 28 and 26 fruit whose dot plots are roughly symmetric with no outliers.
- A 2-SampTTest output reads: \mu_1 > \mu_2, t = 2.2136, p = 0.01647, \text{df} = 38, \bar{x}_1 = 12.1, \bar{x}_2 = 11.4, s_1 = s_2 = 1.0, n_1 = n_2 = 20. Confirm t from the summaries, find the conservative p-value, and decide at \alpha = 0.05 and at \alpha = 0.01.
- Suppose Malee had chosen \alpha = 0.045 for Example 1 before collecting her data. What do the conservative and the technology tests decide, and which should she report?
- Six fruit from each of two orchards, in grams. Orchard E: 10.6, 11.4, 12.1, 11.0, 11.8, 11.5. Orchard F: 10.2, 10.9, 11.3, 10.5, 11.6, 10.4. Neither shows strong skew or outliers. Test whether the mean masses differ at \alpha = 0.05, with conservative and with Welch degrees of freedom.
- For Exercise 2, compute the Welch degrees of freedom and the technology p-value. Does the decision change?
Day 2: experiments, sides and intervals
- Malee assigns 40 trees at random, 20 to a new watering schedule and 20 to the usual one. New: \bar{x} = 12.9 g, s = 1.4 g; usual: \bar{x} = 12.0 g, s = 1.5 g. She expected the new schedule to help. Test at \alpha = 0.05 and conclude in context.
- Redo Exercise 9 with a two-sided alternative. Explain why Malee may not choose between the two versions after seeing the data.
- An experiment reports a 95\% interval for \mu_{\text{new}} - \mu_{\text{usual}} from 0.12 to 1.48 g. What does a two-sided test at \alpha = 0.05 decide? What does a one-sided test of H_a\colon \mu_{\text{new}} - \mu_{\text{usual}} > 0 at \alpha = 0.025 decide? Explain.
- Kittisak bags the fruit clusters on 14 trees chosen at random from 29, leaving 15 unbagged. Bagged: \bar{x} = 11.9 g, s = 0.9 g; unbagged: \bar{x} = 11.3 g, s = 1.2 g. Test H_a\colon \mu_{\text{bag}} - \mu_{\text{none}} > 0 at \alpha = 0.05, then build the 90\% interval (t^* = 1.771 for 13 df) and check that it agrees.
- For each study, say whether rejecting H_0 would show cause, and why. (a) Two fertilizers are assigned at random to trees. (b) Random samples of fruit from hillside and from valley orchards are compared. (c) Trees whose owners chose to install shade nets are compared with a random sample of other trees.
- Chanida records the mean fruit mass of the same 15 trees last season and this season, and wants to test whether it changed. Is the two-sample t test right? Explain.
- Chanida assigns 36 trees at random to a seaweed fertilizer or none, 18 each. Fertilized: \bar{x} = 12.2 g, s = 1.5 g; none: \bar{x} = 11.9 g, s = 1.4 g. Test H_a\colon \mu_{\text{sea}} - \mu_{\text{none}} > 0 at \alpha = 0.05. A neighbor concludes that the fertilizer does nothing. Respond.
Reasoning
- Explain why the conservative p-value is never smaller than the technology p-value for the same data.
- Show that a two-sided test of \mu_1 - \mu_2 = 0 at level \alpha rejects exactly when the confidence interval at level 1 - \alpha, built with the same degrees of freedom, leaves out 0.
- Explain why choosing a one-sided alternative after seeing which way the samples differ overstates the evidence.
- Kittisak runs the same arithmetic on two orchards (Example 1) and on Chanida’s experiment (Example 4). Explain why only one of the two results supports a claim of cause.
- Explain why the normal condition must be checked for each group separately, even when one sample is large.
Answers
- H_0\colon \mu_A - \mu_B = 0, H_a\colon \mu_A - \mu_B \neq 0. \displaystyle \text{SE} = \sqrt{0.000333 + 0.000514} \approx 0.02911, t = 0.05/0.02911 \approx 1.717, \text{df} = 27, p-value \approx 0.0974. Fail to reject: at \alpha = 0.05 there is no convincing evidence that mean peel thickness differs between the orchards.
- H_a\colon \mu_A - \mu_B < 0. \displaystyle \text{SE} = \sqrt{0.005 + 0.00392} \approx 0.09445, t = -0.15/0.09445 \approx -1.588, \text{df} = 17, p-value \approx 0.0653. Fail to reject: no convincing evidence at 0.05 that orchard A’s peels are lighter on average.
- Hypotheses are about population means, not sample means, which are known (11.8 and 10.9). Correct: H_0\colon \mu_A - \mu_B = 0, H_a\colon \mu_A - \mu_B > 0, with \mu_A, \mu_B the mean fruit mass in each orchard’s harvest.
- (a) No: the sample of 10 is small and strongly skewed, and the large sample of 35 cannot rescue it. (b) No: not random; low fruit may differ from the rest. (c) No: 50 is more than a tenth of 400, which is 40. (d) Yes: each small sample has a clean dot plot.
- \displaystyle \text{SE} = \sqrt{0.05 + 0.05} \approx 0.31623, t = 0.7/0.31623 \approx 2.2136, matching. Conservative \text{df} = 19, p-value \approx 0.0196. At 0.05 both p-values reject; at 0.01 both fail to reject.
- Conservative p-value \approx 0.0479 > 0.045: fail to reject. Technology p-value \approx 0.0443 \le 0.045: reject. She reports the method she chose in advance, not whichever she prefers after seeing both; with technology planned, she rejects and says the evidence is borderline.
- \bar{x}_E = 11.4, s_E \approx 0.5404; \bar{x}_F \approx 10.8167, s_F \approx 0.5492. \text{SE} \approx 0.31455, t \approx 1.854. Conservative \text{df} = 5: two-sided p-value \approx 0.123. Welch \text{df} \approx 10.00: p-value \approx 0.093. Both fail to reject: no convincing evidence that the mean masses differ.
- \displaystyle \text{df} = \frac{0.00892^2}{0.005^2/17 + 0.00392^2/19} \approx 34.91, p-value \approx 0.0606. Still above 0.05: fail to reject.
- H_a\colon \mu_{\text{new}} - \mu_{\text{usual}} > 0. Random assignment, 20 per group, normal condition given. \displaystyle \text{SE} = \sqrt{0.098 + 0.1125} \approx 0.45880, t = 0.9/0.45880 \approx 1.962, \text{df} = 19, p-value \approx 0.0323. Reject: convincing evidence that the new schedule causes heavier mean fruit mass on these 40 trees.
- Two-sided p-value \approx 2 \times 0.0323 \approx 0.0646 > 0.05: fail to reject. The alternative must be fixed before the data, since choosing the side after seeing the data halves the p-value with no new evidence.
- Both reject. The 95\% interval leaves out 0, so the two-sided test at 0.05 rejects. The one-sided test at 0.025 matches the interval at 1 - 2(0.025) = 0.95, which lies wholly above 0 on the side H_a predicts, so it rejects too.
- \displaystyle \text{SE} = \sqrt{0.057857 + 0.096} \approx 0.39225, t = 0.6/0.39225 \approx 1.530, \text{df} = 13, p-value \approx 0.0750: fail to reject. Interval 0.6 \pm 1.771 \times 0.39225 = 0.6 \pm 0.6947, from -0.095 to 1.295 g, which contains 0: agreement.
- (a) Yes: random assignment balances everything but the fertilizer, apart from chance. (b) No: location cannot be assigned, and hillside and valley orchards differ in soil, water and more. (c) No: owners chose the nets, so whatever led to that choice is mixed up with the nets.
- No. The same trees appear in both columns, so the two sets are not independent: a tree with heavy fruit one season tends to have heavy fruit the next. Use Paired data: interval and test for a mean difference: the 15 differences, tested with a one-sample t test.
- \displaystyle \text{SE} = \sqrt{0.125 + 0.108889} \approx 0.48362, t = 0.3/0.48362 \approx 0.620, \text{df} = 17, p-value \approx 0.2716. Fail to reject: no convincing evidence that the fertilizer raises mean fruit mass on these trees. The neighbor is wrong: failing to reject does not show no effect; a small effect may be hidden by the large standard error.
- The Welch degrees of freedom are never below the smaller of n_1 - 1 and n_2 - 1. The statistic t is the same either way, and a t curve with fewer degrees of freedom has heavier tails, so it puts at least as much area beyond t.
- The test rejects when |t| > t^*, where t^* cuts off \alpha/2 in each tail. Multiplying by \text{SE} > 0, that is |\bar{x}_1 - \bar{x}_2| > t^*\,\text{SE}, which says the interval (\bar{x}_1 - \bar{x}_2) \pm t^*\,\text{SE} does not reach 0.
- The two-sided p-value is twice the one-sided one in the direction the data fell. Picking that direction after looking guarantees the smaller tail, so the p-value is halved without any new evidence, and H_0 is rejected more often than \alpha when it is true.
- Chanida assigned thinning at random, so apart from chance her two groups differed only in the treatment. Malee’s orchards differ in soil, care, tree age and much else, so a difference in their means cannot be laid on any one cause.
- Each sample mean needs a sampling distribution close to normal. A large sample makes its own mean close to normal, but the mean of a small, strongly skewed sample is still skewed, and so is the difference.