Lecture 15

CentralCentralLimitLimitTheorem,Theorem,ConfidenceConfidenceIntervalsIntervalsforforMeansMeans--LargeLargeSamplesSamples

TheTheCentralCentralLimitLimitTheoremTheorem

Remark

Computing with the normal distribution requires knowledge of the population parameters $\mu$ and $\sigma$. In practice, these parameters are often unknown. In such cases, we use the sample mean $\overline{X}$ and the sample standard deviation $s$ to estimate the population mean and standard deviation. But how good are these estimates?

Let us start with the sample mean $\overline X$. Different random samples will give different sample means. Thus the sample mean $\overline X$ is a random variable as well. What are the properties of this random variable, such as the shape of its distribution, its expected value and its standard deviation? The Central Limit Theorem (CLT) provides the answer to this question.

Example 1

To illustrate the relation between population mean and all sample means consider the following data on the number of hours per week four people have spent watching videos on the internet:$$ \begin{array}{l|c|c|c|c} \text { Person } & \text{ Ann } & \text { Bob } & \text { Cathy } & \text { Dylan } \\ \hline \text { Hours } & 16 & 8 & 4 & 10 \end{array}$$The mean number of videos watched in this (very small) population is$\mu = \frac{16 + 8 + 4 + 10}{4} = 9.5$ hours. From this population we can draw without replacement all possible samples of size $n=2$. The sample means for all possible samples of size 2 are:$$ \begin{array}{l|c|c} \text { Sample } & \text{ Sample Mean } \\ \hline \text { Ann, Bob } & 12 \\ \text { Ann, Cathy } & 10 \\ \text { Ann, Dylan } & 13 \\ \text { Bob, Cathy } & 6 \\ \text { Bob, Dylan } & 9 \\ \text { Cathy, Dylan } & 7 \end{array}$$The mean of all possible sample means is$$ \mu_{\overline X} = \frac{12 + 10 + 13 + 6 + 9 + 7}{6} = 9.5 $$which is the same as the population mean. This is no coincidence. The claim that the mean of all possible sample means is equal to the population mean holds for any sample size and any population distribution. This is one of the assertions of the Central Limit Theorem.

Theorem

For large samples of size $ n > 30$ the distribution of sample means $\overline X$ is approximately normal with$$\mu_{\overline X} = \mu \quad \text{and} \quad \sigma_{\overline X} = \frac{\sigma}{\sqrt{n}}$$

Remark

1. The CLT makes claims about the shape of the distribution of $\overline X$, its expected value and its standard deviation. The first claim is that the distribution of $\overline X$ is approximately bell-shaped. The second claim is that the mean of all possible sample means is equal to the population mean $\mu$. The third claim is that the standard deviation of the distribution of $\overline X$ is equal to the population standard deviation $\sigma$ divided by the square root of the sample size $n$. Thus with a larger sample size $n$ the standard deviation of the distribution of $\overline X$ is smaller and the values of $\overline X$ are closer to the population mean $\mu$.

2. The Central Limit Theorem is one of the most important theorems in statistics. It is the foundation for many statistical procedures. Much more detailed information on the Central Limit Theorem can be found in textbooks and online courses on mathematical statistics.

Example 2

The population of common ladybugs has a mean size of $\mu = 7.4\ mm$ and a standard deviation of $\sigma = 1.5\ mm.$
a) Assuming that the population distribution is approximately normal, what is the probability that a randomly selected ladybug has a size smaller than $7\ mm?$
b) What is the probability that the mean size of a random sample of $50$ ladybugs is smaller than $7\ mm?$

Solution: a)$$P(X < 7) = P\left(\frac{X - \mu}{\sigma} < \frac{7 - 7.4}{1.5}\right) = P(Z < -0.27) = 0.3944$$b) The mean size of a random sample of $50$ ladybugs is a random variable $\overline X$ with a mean of $\mu_{\overline X} = 7.4\ mm$ and a standard deviation of $\sigma_{\overline X} = \frac{\sigma}{\sqrt{n}} = \frac{1.5}{\sqrt{50}} = 0.2121\ mm$.$$P(\overline X < 7) = P\left(Z < \frac{7 - 7.4}{0.2121}\right) = P(Z < -1.89) = 0.0292$$Notice that it is much less likely to find a sample mean of $7\ mm$ or less than to find a single ladybug of size $7\ mm$ or less.

Example 3

The average height of females in Canada is $\mu = 163.8\ cm$ with a standard deviation of $\sigma = 7.1\ cm.$ Compute the $90$th percentiles for a single Canadian female and for the mean height of a random sample of $100.$

Solution: $P_{90} \rightarrow z = 1.28$. For a single female we have$$ X = \mu + z\sigma = 163.8 + 1.28\cdot 7.1 = 173.2\ cm$$For the mean height of a random sample of $100$$$ \overline X = \mu + z\frac{\sigma}{\sqrt{n}} = 163.8 + 1.28\cdot \frac{7.1}{\sqrt{100}} = 164.1\ cm$$

ConfidenceConfidenceIntervalsIntervalsforforMeansMeans--LargeLargeSamplesSamples

Remark

The Central Limit Theorem allows us to get an interval estimate for the population mean $\mu$ based on a value for the sample mean $\overline X$ of a random sample. This interval estimate is called a confidence interval for the population mean. The confidence interval is based on the fact that the distribution of $\overline X$ is approximately normal for large samples, $n>30.$

Let us assume that the sample mean at our disposal is not in the tails of the distribution of $\overline X$, but is close to the center of the distribution. Specifically, we will assume that the sample mean is not within the $\alpha/2$ extreme values (tails) of the distribution of $\overline X$. Here $\alpha$ is a small number, such as $0.05$ or $0.01$. Thus we are assuming that our sample is amongst the $1-\alpha$ of all possible samples of size $n$ which are closest to to the mean.

Observe that the probability that a random sample is in the left $\alpha/2$ tail or right $\alpha/2$ tail of the distribution is $\alpha.$Thus our conclusions will only hold for $1-\alpha$ of all possible samples of size $n$. This will the confidence level of our interval estimate.

Now the probability that our random sample has a sample mean not in the $\alpha/2$ tails of the distribution is:$$P\left(-z_{\alpha/2} < \frac{\overline X - \mu}{\sigma/\sqrt{n}} < z_{\alpha/2}\right) = 1 - \alpha$$A couple of simple algebraic manipulations give us the following inequality:$$P\left(\overline X - z_{\alpha/2}\frac{\sigma}{\sqrt{n}} < \mu < \overline X + z_{\alpha/2}\frac{\sigma}{\sqrt{n}}\right) = 1 - \alpha$$These lower and higher bounds for the population mean $\mu$ in this formula are obtained under the assumption that our sample mean $\overline X$ is not in the tails of the distribution. A $100(1-\alpha)$% confidence interval for the population mean $\mu$ is given by$$\overline X - z_{\alpha/2}\frac{\sigma}{\sqrt{n}} < \mu < \overline X + z_{\alpha/2}\frac{\sigma}{\sqrt{n}}$$where $z_{\alpha/2}$ is the z-score that has an area of $\alpha/2$ to its right. This confidence interval holds for large samples of size $n>30$.

Remark

The formula for the confidence interval uses the population standard deviation $\sigma$ which is rarely known. In practice, we use the sample standard deviation $s$ to estimate the population standard deviation, and this approximation is justified for large samples.

Formula

A $100(1-\alpha)$% confidence interval for the population mean $\mu$ is given by$$\overline X - z_{\alpha/2}\frac{s}{\sqrt{n}} < \mu < \overline X + z_{\alpha/2}\frac{s}{\sqrt{n}}$$where $z_{\alpha/2}$ is the z-score that has an area of $\alpha/2$ to its right. This confidence interval holds for large samples of size $n>30$.

Example 4

A random sample of 120 earthworms has an average length of $16\ cm$ with a standard deviation of $3.5\ cm$. Compute a $95$% confidence interval for the average length of all earthworms.

Now, computing the confidence interval is a simple matter of plugging in the numbers:$$16 - 1.96\frac{3.5}{\sqrt{120}} < \mu < 16 + 1.96\frac{3.5}{\sqrt{120}}$$$$ 14.9 < \mu < 17.1\quad \text{with}\ 95\% \ \text{confidence.}$$

Example 5

Your team caught and banded 48 male American robins and 55 female American robins. Here are the average weights and standard deviations for the two groups:$$\begin{array}{l|c|c} \text{Gender} & \text{Average Weight (g)} & \text{Standard Deviation (g)} \\ \hline \text{Male} & 83 & 3.9 \\ \hline \text{Female} & 75.2 & 3.6 \end{array}$$Construct a $90$% confidence interval for the population average weights.

Males:$$83 - 1.645\frac{3.9}{\sqrt{48}} < \mu < 83 + 1.645\frac{3.9}{\sqrt{48}}$$$$81.7 < \mu < 84.3 \quad \text{with} \; 90\% \; \text{confidence}$$Females:$$75.2 - 1.645\frac{3.6}{\sqrt{55}} < \mu < 75.2 + 1.645\frac{3.6}{\sqrt{55}}$$$$73.9 < \mu < 76.5 \quad \text{with} \; 90\% \; \text{confidence}$$We can pose and answer an additional question: Are the male American robins heavier than the females?

Since there is no overlap between the two confidence intervals, we can conclude that the male American robins are heavier, but only with $90$% confidence.

Example 6

A random sample of $48$ snowy owls has an average wingspan of $137.5\ cm$ with a standard deviation of $9.7\ cm$. Construct $90$%, $95$% and $99$% confidence intervals for the population wingspan length of snowy owls.

The confidence intervals are: $90$%:$$137.5 - 1.645\frac{9.7}{\sqrt{48}} < \mu < 137.5 + 1.645\frac{9.7}{\sqrt{48}}$$$$135.2 < \mu < 139.8 \quad \text{with} \; 90\% \; \text{confidence}$$$95$%:$$137.5 - 1.96\frac{9.7}{\sqrt{48}} < \mu < 137.5 + 1.96\frac{9.7}{\sqrt{48}}$$$$134.8 < \mu < 140.2 \quad \text{with} \; 95\% \; \text{confidence}$$$99$%:$$137.5 - 2.576\frac{9.7}{\sqrt{48}} < \mu < 137.5 + 2.576\frac{9.7}{\sqrt{48}}$$$$133.9 < \mu < 141.1 \quad \text{with} \; 99\% \; \text{confidence}$$Let us draw the three confidence intervals on the same graph:

Notice that the confidence interval becomes wider as the confidence level increases.

Remark

Half the length of the confidence interval is called the margin of error of the estimate. The margin of error is a measure of the precision of the estimate. The margin of error is the value that is added to and subtracted from the sample mean to obtain the lower and upper bounds of the confidence interval.

Confidence intervals obey a no free lunch principle. For a fixed sample size:
1. Higher precision (lower error) implies lower confidence.
2. Higher confidence implies lower precision (higher error).

Lecture 14 Lecture 16