The Central Limit Theorem
Sample from a population that looks nothing like a bell curve. The averages come out as one anyway.
Overview
What the theorem says
Take repeated samples of size n from any population with a finite mean and variance. Compute each sample's mean. Those means have a distribution of their own, and as n grows it approaches a normal distribution — with mean equal to the population mean and standard deviation sigma / sqrt(n).
The population's own shape does not appear in that statement anywhere. It can be skewed, bimodal, discrete, or wildly irregular.
The population above is deliberately unpleasant: two separate humps and a long tail. Nothing about it is bell-shaped. Set n to 1 and the right-hand panel is that same shape, because a sample of one *is* a draw from the population.
Now drag n upward. By 5 the two humps have merged. By 30 the right-hand panel is a clean bell, and the fitted normal curve sits on top of the resampled histogram. Nothing about the population changed.
The Central Limit Theorem
Worth knowing
σ/√n, which is why quadrupling n only halves the uncertainty.The Central Limit Theorem
Why the normal distribution turns up everywhere, stated precisely enough to know when it does not.
Three things it is not
Not "large samples are normally distributed." The sample is not normal; it looks like the population, because that is what it is drawn from. It is the *mean* that goes normal.
Not "everything is normal." Heights are roughly normal because they are the sum of many small independent influences. Incomes are not normal and no amount of data makes them so.
Not a licence to ignore the population. It describes the sampling distribution of a statistic, not the data.
The square root
The standard deviation of the sample mean — the standard error — is sigma / sqrt(n). The square root is the practical heart of the theorem.
To halve your uncertainty you must quadruple your sample. To divide it by ten you need a hundred times the data. Precision gets expensive fast, and this single fact governs how large a survey has to be, how long an A/B test must run and why polls stubbornly report margins of around three percent.
[The sampling distribution module](sampling_distributions.html) plots that decay directly.
How large is large enough
The usual rule of thumb is n = 30, and like all such rules it is a summary of something more specific.
Near-symmetric population: convergence is fast, and 10 may do.
Strongly skewed: 30 is not enough. Heavily skewed populations can need hundreds before the mean is convincingly normal.
Heavy tails: worse still, and there is a limit case below.
The honest procedure is the one on this page: resample and look. Drag n and find where the histogram stops looking lumpy for *your* population.
When it fails outright
The theorem requires a finite variance, and that condition is not decorative.
The Cauchy distribution has none — nor a finite mean. Average n Cauchy draws and you get another Cauchy, with exactly the same spread as one draw. No amount of averaging concentrates anything. The sample mean of a million draws is no better than the first.
Heavy-tailed distributions that do have a finite variance still converge, but slowly enough to matter. Financial returns are the standard cautionary example: methods that assume normality of averages understate the chance of extreme outcomes, because convergence has not really happened at the sample sizes in use.
Where it shows up
Confidence intervals and t-tests assume the sampling distribution of the mean is normal. That assumption is this theorem.
A/B testing: a conversion rate is a mean of Bernoulli draws.
Bootstrapping exists partly to avoid needing it when it does not hold.
Measurement error is often modelled as normal because it is the sum of many small independent errors.
Standardising a sample mean
The theorem is used, in practice, to turn an observed sample mean into a probability, and the arithmetic is short. Suppose a population has mean 50 and standard deviation 8, and you take a sample of n = 16. By the theorem the sample mean is approximately normal, centred at 50, with standard error 8 / √16 = 2.
Now you observe a sample mean of 54. How surprising is that?
z = (54 − 50) / 2 = 2.0
The observed mean is two standard errors above the population mean, and for a normal distribution about 2.3% of samples fall that far above by chance alone. That single z is the engine under a one-sample z-test and a confidence interval alike: the interval 54 ± 1.96 × 2 is [50.08, 57.92], which — just barely — excludes 50. Note which quantity did the work: the standard error, not the population's own spread. The raw data has sd 8, but the *mean* of sixteen draws has spread 2, and it is that shrinkage by √n that makes an observed difference of 4 look like a lot rather than a little.
Increasing the sample sharpens that verdict fast, through the same square root. At n = 64 the standard error falls to 8 / 8 = 1, so the same observed mean of 54 now sits z = 4 standard errors out — a result that was mildly surprising at sixteen samples becomes overwhelming at sixty-four. Nothing about the population changed; the larger sample simply pins the mean down four times more tightly than the raw data's spread of 8 would suggest, and that shrinkage is the entire practical content of the theorem.
Where it goes wrong
Applying it to the data instead of the statistic. Extremely common.
Trusting n = 30 on skewed data. Check.
Assuming it applies to the maximum or the variance. It is about sums and means. Extreme values have their own limit theory, and a different family of limiting distributions.
Ignoring dependence. The classical version needs independent draws. Time series data usually is not independent, and the effective sample size is smaller than the count.
Watch it happen
A deliberately lopsided population, and the means of samples drawn from it. The spread of those means shrinks like one over the square root of the sample size.
Check yourself
0 of …Answer without scrolling back up.
What exactly becomes normal?
You want to halve your uncertainty about a mean. What must you do?
For which population does the theorem fail entirely?
The Central Limit Theorem
Take repeated samples of size n from any population with a finite mean and variance. Compute each sample's mean. Those means have a distribution of their own, and as n grows it approaches a normal distribution — with mean equal to the population mean and standard deviation sigma / sqrt(n).