How You See

Chapter 4 When the inference goes wrong

Optical illusions, taken apart

An illusion is your visual system applying a rule that is usually right in a place where it is wrong. Each one here has a control that exposes the rule.

8 min read · 5 interactive figures · 3 questions

Two lines, one with arrowheads and one with tails. Drag the end of the lower line (or use the slider) until the two shafts look equal to you, then press Reveal to see how far off your setting was.

An optical illusion is a mismatch between what you see and what is physically there. Most illusions happen because the visual system does not measure the image on the retina; it uses that image to infer the world, relying on assumptions that are usually right: that shadows darken surfaces, that converging lines recede, that a steady scene is not moving. An illusion is a picture built where an assumption fails. That makes illusions one of the best tools vision science has for finding out what the assumptions are.

The four illusions on this page each come with a control that exposes the rule behind it. One warning before starting: for several famous illusions, the explanation in textbooks is not settled. Each section says how strong the evidence is.

The Müller-Lyer illusion

The figure at the top is the most studied illusion of all. Two equal lines, one ending in arrowheads and one in arrow tails, do not look equal: the line with tails looks longer[1]. Rather than take that on trust, measure it. Adjust the lower line until the two look the same length to you, and only then press Reveal.

Question 1 Prediction: does the illusion survive knowing the answer?#
Figure 1 The two figures with equal shafts. Look at them now that you know. Then change the fin angle to see whether the effect grows or shrinks for you, untick Show the fins to see the bare shafts, and use Reveal to mark where the upper shaft ends.#

Why does it happen? The best-known answer is Richard Gregory's. In 1963 he proposed that the figures are read as corners in three dimensions: arrow tails as the inside corner of a room, receding away from you; arrowheads as the outside corner of a building, jutting toward you. A line that seems farther away but casts the same-sized image must be longer, so the visual system scales it up. Gregory called this "inappropriate constancy scaling"[2]. contested

It is a beautiful idea and it has trouble. The effect persists when the lines end in squares or other adornments that suggest no corners, and even in figures made only of dots, which undermines explanations based on what arrowheads and tails might signify[1]. Catherine Howe and Dale Purves offered a statistical alternative in 2005: by measuring a database of natural scenes with range-finders, they showed that the physical sources most likely to have produced each figure's image differ in length, in the direction people see[1]. They concluded that the illusion is a byproduct of the brain's probabilistic strategy for coping with images whose physical source is uncertain, and they note that Gregory's intuition pointed in the right general direction[1]. Neither account is universally accepted, and the Müller-Lyer remains, in Howe and Purves' words, the most controversial of the classical geometric illusions. contested

The checker-shadow illusion

Here the evidence is clearer. Edward Adelson, a vision scientist at MIT, published the original version of this figure in 1995[3]; the one below is drawn from scratch by this site, from the physics of light. A checkerboard of light and dark squares sits under a soft shadow cast by a green cylinder. Square A is a dark square in full light. Square B is a light square in the shadow.

Question 2 Prediction: which square sends more light to your eye?#
Figure 2 Square A (a dark check in full light) and square B (a light check in the shadow). The readout gives their screen values, which are identical: the dark checks reflect a quarter as much light as the light ones, and the shadow lets through a quarter of the light. Tick Hide everything but A and B or Join A and B to see it. Move the shadow slider and B changes physically, while A does not.#

The light that reaches your eye from a surface is the product of two things: how much light falls on it and how much of that it reflects. The retina receives only the product. To see the surface's own lightness, which is what matters for recognizing objects, the visual system has to divide out the illumination, and it estimates the illumination from context: the soft edge of the shadow, the cylinder that casts it, the regular pattern of the checks. Its estimate here is excellent. It correctly infers that B is a light square in shadow. What it reports to you is that inference, not the light. documented

This is why the illusion does not fade when you know the answer, and why the checker-shadow figure is less an error than a success seen from the wrong angle. A visual system that reported raw light would tell you that a white page in shadow is gray and a dark coat in sunlight is pale.

The Hermann grid

Black squares separated by white streets. Look around the grid: faint gray smudges appear at the crossings you are not looking at directly, and disappear from the one you fixate.

For decades textbooks explained this with the retina. A ganglion cell with an "on-center, off-surround" receptive field is excited by light in its center and inhibited by light around it. Centered on a crossing, it has white on four sides of its surround; centered on a street, only on two. More inhibition at the crossing, a weaker signal, a darker smudge. The account goes back to Baumgartner in 1960[5]. contested

Question 3 Prediction: what should wavy streets do to the smudges?#
Figure 3 A Hermann grid with straight streets. Let your gaze wander to see the smudges. Then raise Waviness of the streets: for most people the smudges fade and vanish, even though each crossing is still a white patch of about the same size. Change the street width as well.#

The wavy grid comes from János Geier and colleagues, who showed in 2008 that small distortions of the street edges abolish the illusion while leaving the conditions of the retinal explanation unchanged; they concluded that the straightness of the edges is the main factor[6]. Peter Schiller and C. E. Carvey had reached the same verdict three years earlier by other routes, adding that the actual layout of ganglion cell receptive fields is not what the theory assumes. They proposed instead that the effect arises in a class of simple cells in the primary visual cortex[5]. contested

So the Hermann grid teaches two lessons. One is about vision: the smudges probably arise after the retina, in cortical mechanisms that analyze edges and lines. The other is about explanations. A tidy account can be repeated in textbooks for half a century before a simple manipulation, the kind you have just made, shows it does not work.

The motion aftereffect

Stare at a waterfall for a minute, then look at the rocks beside it: they seem to drift upward. This is the motion aftereffect, described in print by Addams in 1834[7]. Here the explanation is well supported. Neurons in the visual cortex are tuned to directions of motion. Prolonged motion in one direction reduces the responsiveness of the cells tuned to it, so that when the motion stops, the balance between detectors tuned to opposite directions is tipped, and the competition produces a false signal of motion the other way[8]. established

Figure 4 A spiral that rotates for 30 seconds when you press Start. Keep your eyes on the red dot at its center. When it stops, the still spiral appears to turn the other way, or to shrink or swell; then glance at a textured surface such as the back of your hand. Nothing flickers. You can change the watching time and the speed.#

The adaptation is not one simple "fatigue". Anstis, Verstraten and Mather's review concludes that it happens at several sites in the cortex and is better thought of as the visual system recalibrating itself, a form of error correction or efficient coding, than as cells getting tired[8]. documented The striking part is the paradox: the rocks seem to move, yet they do not seem to go anywhere. Motion and position are computed largely by different machinery, and the aftereffect fools one far more than the other: it shifts where things seem to be only slightly[9].

What the illusions have in common

Line up the four. In the checker-shadow figure, the brain divides out the illumination it infers. In the Müller-Lyer, it may be scaling for a depth it infers, or betting on the likeliest physical source. In the motion aftereffect, it is recalibrating its motion detectors. Only the Hermann grid still has no agreed story, and the classic one has been seriously challenged.

None of these is a fault in the eye. The optics in the first chapter deliver an image; the receptors in the second sample it; everything you see beyond that is the brain's best account of what is out there. The blind spot is filled with a guess, color is a ratio, depth is a set of bets, and lightness is an estimate. Illusions are what those processes look like when you can catch them in the act.