Central Limit Theorem

One property of the normal distribution is that the sum of n independent random variables$X_{1},X_{2},\dots ,X_{n}$ with Normal distribution, is also normally distributed. This property remains true for any value of n. If the random variables $X_{1},X_{2},\dots ,X_{n}$ are not normally distributed, then this property is not exactly true, but it remains approximately correct for large n. Let $X_{1},X_{2},\dots ,X_{n}$ be independently and identically distributed random variables with E($X_{i}$ ) = $\mu$ and Var($X_{i}$ ) = $\sigma ^{2}>0$ for i = 1,$\dots$ ,n. Then the sum of these random variables is for large n approximately normally distributed:E($X_{1}+X_{2}+\dots +X_{n}$ ) = n$\mu$ and Var($X_{1}+X_{2}+\dots +X_{n}$ ) = n$\sigma ^{2}$ $X_{1}+X_{2}+\dots +X_{n}\approx$ N(n$\mu$ ,n$\sigma ^{2}$ ),where $\approx$ means approximately for large n. Let $X_{1},X_{2},\dots ,X_{n}$ be independently and identically distributed random variables with E($X_{i}$ ) = $\mu$ and Var($X_{i}$ ) = $\sigma ^{2}>0$ for i = 1,$\dots$ ,n. Then the mean of these random variables is for large n approximately normally distributed:E$\left({\frac {1}{n}}(X_{1}+X_{2}+\dots +X_{n})\right)$ =E$\left({\overline {x}}\right)$ = $\mu$ and Var$\left({\overline {x}}\right)$ = ${\frac {\sigma ^{2}}{n}}$ ${\overline {x}}\approx$ N($\mu$ ,${\frac {\sigma ^{2}}{n}}$ ),where $\approx$ means approximately for large n This result requires that none of the random variables are responsible for most of the variance.The distribution N($\mu ,{\frac {\sigma ^{2}}{n}}$ ) depends on the number of the summands n and for infinite n it would have infinite and infinite variance. The meaning of this theorem can be described more clearly if we use standardized sums of random variables.
Let $X_{1},\dots ,X_{n}$ be independent and identically distributed random variables: E($X_{i}$ ) = $\mu$ and Var($X_{i}$ ) = $\sigma ^{2}$ > 0 . Then the distribution function $F_{n}(z)=P(Z_{n}\leq z)$ of ${\overline {x}}$ $Z_{n}={\frac {\sum \limits _{i=1}^{n}{\frac {X_{i}}{n}}-\mu }{\sqrt {\sigma ^{2}/n}}}={\frac {1}{\sqrt {n}}}\sum _{i}{1}^{n}{\frac {{\frac {X_{i}}{n}}-\mu }{\sigma }}$ converges as n$\rightarrow \infty$ to a standard normal distribution: $\lim _{n\rightarrow \infty }F_{n}(z)=\Phi (z)$ The “standardized ” random variable $Z_{n}$ is approximately distributed as a standard Normal distribution: $Z_{n}\approx N(0;1)$ . In this example, we will try to illustrate the principle of the Central Limit Theorem. Let us consider continuous random variables $X_{1},X_{2},\dots$ random variables which are independently and identically uniformly distributed on the interval $[-0,5;0,5]$ : $f(x)=\left\{{\begin{array}{ll}1\quad &{\text{for}}\ -0.5\leq x\leq 0.5\\0\quad &{\text{otherwise.}}\end{array}}\right.$ The expected value and the variance are: $E(X)={\frac {b+a}{2}}={\frac {0.5-0.5}{2}}=0$ $Var(X)={\frac {(b-a)^{2}}{12}}={\frac {[0.5-(-0.5)]^{2}}{12}}={\frac {1}{12}}\,.$ Let us consider a sequence of the sum of these variables; the index of the variable $Y$ denotes the number of observations in the sample: $Y_{n}=\sum _{i=1}^{n}X_{i}\qquad n=1,2,3,\dots \,.$ For example, for $n=1$ , $n=2$ , and $n=3$ we get:$Y_{1}=X_{1}$ $Y_{2}=X_{1}+X_{2}$ $Y_{3}=X_{1}+X_{2}+X_{3}$ . and the densities: $f(y_{1})=\left\{{\begin{array}{ll}1\quad &{\text{for}}\ -0.5\leq y_{1}\leq 0.5\\0\quad &{\text{otherwise}}\end{array}}\right.$ $f(y_{2})=\left\{{\begin{array}{ll}1+y_{2}\quad &{\text{for}}\ -1\leq y_{2}\leq 0\\1-y_{2}\quad &{\text{for}}\ 0\leq y_{2}\leq 1\\0\quad &{\text{otherwise}}\end{array}}\right.$ $f(y_{3})=\left\{{\begin{array}{ll}0.5(1.5+y_{3})^{2}\quad &{\text{for}}\ -1.5\leq y_{3}\leq -0.5\\0.5+(0.5+y_{3})(0.5-y_{3})\quad &{\text{for}}\ -0.5 All these densities are plotted in the following figure, which also contains a plot of a $N(0,1)$ density for comparison:
The convergence towards of these distributions to a Normal density can be clearly seen. As the number of observations increases the distribution becomes more similar to a Normal distribution. In fact, for $n\geq 30$ we can hardly see any differences. The (Lindeberg and Lévy) Central Limit Theorem is the main reason that the Normal distribution is so commonly used. The practical usefulness of this theorem derives from the fact that a sample of identically distributed independent random variables has an approximately Normal distribution as the sample increases (usually $n\geq 30)$ This theorem becomes particularly important when the deriving the sampling distribution of the statistics. The convergence towards the Normal distribution will be very quick if the distribution of the random variables is symmetric. If the distribution is not symmetric, then the convergence will be much slower. The Central Limit Theorem has many various generalizations (e.g. Lyapunov CLT for independent, but not identically distributed random variables). Furthermore, there are also limit theorems that describe convergence towards other sorts of distributions.