Bernoulli, Binomial and Poisson
One trial, then n trials, then the limit as n runs away. Three distributions built from the same coin.
Overview
Bernoulli: the atom
One trial, two outcomes, one parameter. Success with probability p, failure with probability 1 - p.
Mean p, variance p(1-p). That variance is worth a moment: it is largest at p = 0.5 and falls to zero at either end, which says something obvious once stated — an event that always happens, or never does, is not random at all. Drag p to 0.02 with Bernoulli selected and watch the variance in the readout collapse.
Every coin flip, click-or-not, churn-or-not and pass-or-fail is a Bernoulli trial, and it is the atom the other two are assembled from.
Bernoulli, Binomial and Poisson
Worth knowing
Bernoulli, Binomial and Poisson
The three distributions behind every count, and how each is built from the one before it.
Binomial: counting the successes
Run n independent Bernoulli trials with the same p and count the successes. That count is binomial.
P(k successes) = C(n, k) * p^k * (1-p)^(n-k)The three factors read directly: p^k for the successes, (1-p)^(n-k) for the failures, and C(n, k) for the number of orders they could have arrived in.
Mean np, variance np(1-p) — both straight from [the rules for combining](expectation_and_variance.html) n independent Bernoulli variables.
Drag n upward and the shape becomes recognisably bell-like. That is [the central limit theorem](central_limit_theorem.html) arriving: a binomial is a sum of independent variables, so it must tend to normal. It is why the normal approximation to the binomial exists, and why it needs np and n(1-p) both comfortably above about 10 — near the edges the binomial is skewed and the symmetric normal fits badly.
The independence and constant-p assumptions are load-bearing. Ten coin flips are binomial. Ten cards drawn without replacement are not, because p changes as the deck depletes; that is the hypergeometric distribution.
Poisson: the limit
Now let n grow and p shrink together, keeping np = lambda fixed. Many trials, each very unlikely, with a stable expected count.
The binomial converges to the Poisson:
P(k) = e^(-lambda) * lambda^k / k!n and p have vanished; only their product survives. That is what makes it useful. You rarely know how many opportunities there were for an event to occur — how many people *could* have visited the site this minute — but you can measure the rate.
So Poisson models counts in a fixed window: arrivals per minute, defects per batch, mutations per genome, goals per match.
Set the distribution to Poisson and compare against a binomial with n = 60 and p = 0.05. Both have mean 3, and the shapes are nearly identical. That is the limit at work.
The Poisson's signature, and its trap
A Poisson has mean and variance both equal to lambda. One parameter fixes both, and there is no way to have a Poisson with mean 3 and variance 10.
Real count data very often has variance well above its mean — overdispersion. Website visits cluster, defects come in bad batches, accidents cluster around conditions. The Poisson assumes events are independent and the rate is constant, and clustering violates both.
Overdispersion is a finding, not an annoyance: it says something is varying that the model treats as fixed. The usual response is a negative binomial, which adds a second parameter and lets the rate itself be random.
Where they show up
A/B testing. Conversions are binomial; the test is about two p values.
Queueing. Poisson arrivals are the standard assumption behind almost all queueing theory.
Rare-event modelling. Failures, fraud, defects.
Naive Bayes for text. Multinomial and Bernoulli variants correspond exactly to counting words and to noting presence or absence.
Class imbalance. A rare positive class is a small p, and the variance of your estimate of it is p(1-p)/n — which is why rare classes need so much more data before a rate estimate settles.
The binomial formula, worked
The formula is best trusted by evaluating it. Five trials, success probability 0.3, P(k) = C(5, k) · 0.3^k · 0.7^(5−k):
| k | C(5, k) | P(k) |
|---|---|---|
| 0 | 1 | 0.1681 |
| 1 | 5 | 0.3601 |
| 2 | 10 | 0.3087 |
| 3 | 10 | 0.1323 |
| 4 | 5 | 0.0283 |
| 5 | 1 | 0.0024 |
The most likely single outcome is one success, not the mean of 1.5, because the distribution is skewed at p = 0.3. The probabilities sum to exactly 1 — they must, since some number of successes between 0 and 5 always happens — and the mean Σ k·P(k) works out to np = 1.5. The three factors each do one job: 0.3^k for the successes, 0.7^(5−k) for the failures, and C(5, k) for the number of orders in which those successes could have arrived. Drop the C(5, k) and you would have the probability of one *specific* sequence; it is there because you asked how many successes, not which trials.
Push n up and p down together, holding np fixed, and this same formula slides into the Poisson: the individual n and p vanish and only their product survives, which is why Poisson models counts when you know the rate but not the number of chances — arrivals per minute, defects per batch. The signature to check is that a Poisson forces mean and variance equal; real counts often have variance well above the mean (overdispersion), and that gap is a finding, not noise — it says something the model treats as fixed is actually varying.
When overdispersion is real, the usual fix is a negative binomial, which adds a second parameter and lets the rate itself vary rather than forcing mean and variance to be equal — the Poisson's rigidity is exactly what it relaxes.
Where it goes wrong
Applying a binomial to sampling without replacement. p changes; use the hypergeometric.
Assuming Poisson without checking dispersion. Compare the sample variance against the sample mean before anything else.
Using the normal approximation near p = 0 or 1. The binomial is skewed there; the approximation is not.
Forgetting the fixed window. A Poisson rate is per unit of something. Double the window and lambda doubles.
The three, side by side
One trial, n trials, and counts with no fixed n. The last block shows a binomial with large n and small p turning into a Poisson.
Check yourself
0 of …Answer without scrolling back up.
Why is a Bernoulli variance largest at p = 0.5?
What survives when a binomial becomes a Poisson?
Your count data has mean 4 and variance 12. What does that suggest?
Bernoulli, Binomial and Poisson
Mean p, variance p(1-p). That variance is worth a moment: it is largest at p = 0.5 and falls to zero at either end, which says something obvious once stated — an event that always happens, or never does, is not random at all. Drag p to 0.02 with Bernoulli selected and watch the variance in the readout collapse.