Chapter 2 The bell curve
The normal curve, measured in standard deviations
A z-score counts standard deviations from the mean; on a normal curve it turns exactly into an area, a percentile or a probability.
On this page
A z-score is the number of standard deviations a value lies above or below the mean: z = (x − μ)/σ. On a normal distribution, every z-score corresponds to an exact area under the curve, and that area is the fraction of the population below the value. A z of 1 puts you at the 84th percentile, a z of 2 at the 97.7th, and a z of −1.96 cuts off the bottom 2.5 %.
The bell-shaped normal curve is the most famous picture in statistics, and most of its fame is earned. It describes the sum of many small independent effects, which is why it appears in measurement errors, in heights within a population and, above all, in the behavior of averages, the subject of Chapter 4. This chapter is about using it: turning values into areas and areas into values, and knowing when the curve is the wrong ruler.
Probability is area under the curve
A normal distribution is described completely by two numbers, its mean μ, which sets the centre, and its standard deviation σ, which sets the width. Its density is
The height of the curve is not itself a probability. A density tells you how crowded the values are near x; probabilities come from areas. The fraction of the population between any two values is the area under the curve between them, and the total area is exactly 1[1]. Drag the cut point in the figure at the top of the page, or type a value, mean and standard deviation.
Try the familiar landmarks. Set μ = 0 and σ = 1 (the standard normal): the area below 1 is 0.8413, below 2 it is 0.9772, and between −1.96 and +1.96 it is 0.9500, which is where the 1.96 in every 95 % confidence interval comes from. Then put the mean back to 100 and the standard deviation to 15, the scale on which many intelligence tests are normed, and ask for the area above 130. The answer, 0.0228, is the same as the area above z = 2, because 130 is two standard deviations above 100.
That is the whole point of the z-score. Every normal distribution is the standard normal, stretched and shifted. Converting a value to z = (x − μ)/σ undoes the stretch and the shift, so one curve, and one table, serves every normal distribution there is:
Here Φ is the cumulative distribution function of the standard normal, the area to the left of z. It has no formula in elementary functions. It is defined by the integral of equation (2.1), and computed with series and continued fractions; the figures on this site evaluate it to about fourteen significant digits, and keep that relative precision far into the tails, where a naive 1 − Φ would round to zero[2].
From a percentile back to a value
The question often runs the other way: what score puts someone in the top 10 %? You need the z whose left-hand area is 0.90, which is z = 1.2816, and then the value x = μ + zσ. On the 100-and-15 scale that is 119.2. Type 90 into the percentile box above to check. The function that does this is the inverse of Φ, sometimes written Φ−1 and sometimes called the normal quantile function or probit.
How much lies within one, two, three standard deviations?
Three areas are worth remembering. About 68 % of a normal population lies within one standard deviation of the mean, about 95 % within two, and about 99.7 % within three. The exact values are 68.27 %, 95.45 % and 99.73 %. R. A. Fisher put the same facts in terms of rarity in 1925: a deviation beyond one standard deviation “occurs about once in three trials. Twice the standard deviation is exceeded only about once in 22 trials, thrice the standard deviation only once in 370 trials”[3]. Those are two-sided counts, and they are correct: 1/0.0455 ≈ 22 and 1/0.0027 ≈ 370. established
Those three numbers belong to the bell curve. Before switching shapes in the next figure, test whether they travel.
Switch shapes and the rule breaks in instructive ways. A flat (uniform) population has nothing beyond 1.73 standard deviations, so 100 % lies within two, but only 57.7 % within one. An exponential population, like waiting times, has 95.0 % within two standard deviations, close to the normal by coincidence, but its outliers all lie on one side, and 98.2 % within three. The long-tailed lognormal shape has more than the normal within one standard deviation and a thin scatter of extreme values far beyond three, which is why the 68–95–99.7 rule can understate how often a long-tailed process produces surprises.
The rule that never fails is Chebyshev’s: at least 1 − 1/k² of any distribution with a finite variance lies within k standard deviations, at least 75 % within two and at least 88.9 % within three. It is weak because it has to cover the worst possible shape. For distributions with a single peak there is a sharper guarantee, the Vysochanskij–Petunin inequality, from which Pukelsheim derived the “three sigma rule” that at least 95 % of any unimodal distribution lies within three standard deviations of its mean[4]. established
A standard normal table, computed exactly
Before computers, every statistics textbook ended with a printed table of Φ, and many courses still teach students to read one. The table below is computed rather than copied, to four decimals in each cell. Rows give z to one decimal place, columns add the second decimal: Φ(1.96) is in the row 1.9 and the column 0.06. For negative z, the columns move further from zero, so −1.96 is the row −1.9 and the column 0.06, and the value there, 0.0250, is 1 − Φ(1.96) by the curve’s symmetry[5].
Most mistakes with z tables are about which area the table reports. This one, like most, reports the area to the left. For “above z” you want 1 minus the cell. For “between −z and z” you want 2Φ(z) − 1. For a two-sided p-value you want twice the smaller tail. The figure at the top of the page does all four directly, without a table, and the full z table has both halves printed for reference.
What z-scores are good for
A z-score puts measurements on a common scale. A child’s height and weight are measured in different units, but each can be expressed as a z-score against reference data for children of the same age and sex, and then the two can be compared or tracked over time. A laboratory value two standard deviations from the reference mean is flagged as unusual. A test statistic in a hypothesis test is often a z-score: the distance between what you observed and what the null hypothesis predicts, measured in standard errors, which is where Chapter 6 picks up the story.
Two cautions keep z-scores honest. First, a z-score is only as good as the mean and standard deviation it uses. If those come from a small sample, the z-score is itself uncertain, and the right reference curve is Student’s t rather than the normal (see Chapter 5). Second, converting a z-score to a percentile with Φ assumes the population is normal. A z of 2 always means “two standard deviations above the mean”; it means “the 97.7th percentile” only when the shape is normal.
Where the curve came from
The normal curve entered mathematics in 1733, when Abraham de Moivre found it as an approximation to the binomial distribution of coin tosses, the story of Chapter 3. Laplace generalized de Moivre’s result into the first versions of the central limit theorem, and Gauss derived the same curve in 1809 as the law of errors that makes the average of observations the most probable value[6]. The name “normal” became standard through Galton and Pearson late in the nineteenth century. It was never meant to imply that other distributions are abnormal, though it has been read that way ever since, as Pearson himself regretted in 1920[7]. documented