Where the shapes come from

The binomial distribution counts successes in a fixed number of independent tries with the same chance each time: heads in 20 coin flips, defective parts in a batch of 50. The Poisson distribution counts events that have many chances to happen but rarely do: typos on a page, calls to a help line in a minute. Both are shapes that a simple random process leaves behind, and both turn into the normal curve when the counts get large.

Statistics courses often present distributions as a catalogue of formulas to look up. They are better understood as the fingerprints of mechanisms. If you know how the randomness works, you can usually say which shape it will make, and when the data refuse to take that shape, the mismatch tells you something about the mechanism. This chapter builds the two most important counting distributions from their mechanisms, and shows the normal curve emerging from both.

A board full of coin flips§

Francis Galton built a board that he described in his 1889 book Natural Inheritance: shot poured through a funnel at the top falls through rows of pins, at each pin is knocked left or right, and collects in compartments at the bottom^[1]. Galton built it to show how a bell-shaped pile arises from many small independent deflections. It is the clearest physical picture of the binomial distribution there is, and the board at the top of this page is a working one. (There is more on the board itself, and its history, on the Galton board page.)

Each ball makes n independent left-or-right choices, so its bin, the number of rights, is a binomial count. The chance of landing in bin k is the number of paths that reach it times the probability of each path:

C(n, k) = n! / (k! (n − k)!) counts the ways to choose which k of the n pins send the ball right. With 12 rows there is exactly one path to each edge bin and 924 paths to the centre bin, which is why the middle fills first. The mean bin is np, and the variance is np(1 − p): each bounce contributes p(1 − p) to the variance, and the variances of independent bounces add^[2]. So the spread grows more slowly than the board: doubling the rows doubles the variance and multiplies the spread by only √2, from 1.73 bins with 12 rows to 2.45 with 24. Set the board to 24 rows and read the spread under it. established

Now tilt the board. With p = 0.2 and 12 rows the pile leans hard to the left and is visibly lopsided. Raise the rows to 24 and it straightens; the skewness of a binomial count is (1 − 2p)/√(np(1 − p)), which shrinks as n grows. Switch on the normal approximation and watch it improve with every row you add.

De Moivre’s discovery§

That improvement is a theorem. In 1733, working on gambling problems with many coin tosses, Abraham de Moivre showed that for large n the binomial probabilities follow a bell curve with mean np and variance np(1 − p)^[3]. It was the first appearance of the normal curve in mathematics, eighty years before anyone used it for errors of measurement, and it is a special case of the central limit theorem of Chapter 4: a binomial count is a sum of n independent zeros and ones. documented

Exact binomial probabilities§

The calculator below (also on its own binomial calculator page) evaluates equation (3.1) exactly, summing it where you ask for a range, and draws the result. It answers questions like “in 20 tries at a 30 % chance, what is the probability of at most 7 successes?” (0.7723) or “of exactly 6?” (0.1916).

The comparison shows why textbooks insist on the continuity correction. A binomial count lives on whole numbers, and the bar for k covers the interval from k − ½ to k + ½. Approximating P(X ≤ 7) by the normal area below 7 throws away half of the bar at 7; the area below 7.5 keeps it. For 100 fair coin flips, the exact chance of 45 to 55 heads is 0.7287. The normal area from 44.5 to 55.5 gives 0.7287; from 45 to 55, without the correction, it gives 0.6827, off by nearly five percentage points. The old rule of thumb for using the approximation at all is that both np and n(1 − p) should be at least 5 or 10, but with exact calculation available there is rarely a reason to approximate^[4]. established

Rare events: the Poisson distribution§

Many counts have no obvious number of tries. How many typing errors on this page? How many meteors in an hour? There are a great many places where a typo could occur and a tiny chance at each. Take a binomial with n chances, each with probability p = λ/n, so that the expected count λ stays fixed, and let n grow. The limit is the Poisson distribution^[5]:

With only 10 chances at 20 % each, the binomial is visibly narrower than the Poisson: it cannot exceed 10, and its variance np(1 − p) = 1.6 is less than its mean of 2. With 1,000 chances at 0.2 % each, the two are indistinguishable at the resolution of the figure. Lucien Le Cam proved a clean bound on the gap: the total variation distance, the largest possible difference in the probability the two distributions give to any set of outcomes, is at most np², which is λ²/n^[6]. The Poisson distribution has one striking property you can read from its formula: its variance equals its mean, both λ. A count whose variance is much larger than its mean is not Poisson, and usually that means the events are not independent: they cluster. established

The Prussian army’s horse kicks§

The most famous Poisson data set is a record of deaths. In 1898 the economist and statistician Ladislaus von Bortkiewicz published Das Gesetz der kleinen Zahlen, “the law of small numbers”, a study of rare events that included the number of soldiers killed by a horse kick in each of fourteen corps of the Prussian army in each year from 1875 to 1894. In the version usually reproduced, ten corps over twenty years give 200 corps-years and 122 deaths in all^[7]. Before looking at how those deaths were spread out, a prediction.

The record: 109 corps-years with no deaths, 65 with one, 22 with two, three with three and one with four^[7]. documented

The best-fitting rate is the average number of deaths per corps-year, 122 deaths in 200 corps-years or 0.61, and at that rate the Poisson predicts 108.7, 66.3, 20.2, 4.1 and 0.6 corps-years. The fit is remarkable, and the data’s variance, 0.61, matches its mean as a Poisson process requires. Nothing about horses or cavalry went into the prediction: only the assumption that each soldier, each year, had a small and independent chance of a fatal kick. That independence is the substance of the model and also its weakness, which Quine and Seneta examine in detail: some corps were larger than others, so the rate cannot have been exactly the same in every corps-year^[7].

The normal curve, again§

Set λ to 12 in the Poisson figure and the ticks form a nearly symmetric bell around 12 with a standard deviation of √12 ≈ 3.5. Set the Galton board to 24 rows and the pile is a bell too. Both are sums: a binomial count is a sum of independent zero-or-one trials, and a Poisson count with a large mean is a sum of many Poisson counts with small means. The central limit theorem says that sums of many independent pieces, whatever the shape of the pieces, end up with the same normal shape. The next chapter tests that claim on populations far stranger than coin flips.