How You See

Chapter 3 Monocular and binocular cues

Depth perception

The image on each retina is flat. Depth is an inference from clues in that image and from the small differences between your two eyes' views.

8 min read · 4 interactive figures · 2 questions

A row of identical posts standing at different distances. Every pictorial depth cue is on. Switch them off one at a time and see how much depth is left; drag the scene sideways to move your viewpoint (motion parallax).

Depth perception is the ability to see the world in three dimensions and judge how far away things are. The image on each retina is flat, so depth has to be inferred. The brain uses monocular cues, which work with one eye (occlusion, relative size, height in the visual field, linear perspective, texture gradients, aerial perspective and motion parallax), and binocular cues, which need both eyes, above all retinal disparity: the small differences between the two eyes' views[1].

The scene at the top of the page is drawn by a pinhole projection of real geometry: seven identical posts stand on flat ground at distances from 4 to 45 meters, and two rails run away from you. Every cue is on. Each checkbox removes one cue by drawing the scene as if that cue carried no information. Try removing them in turn, and then all at once.

A flat image of a deep world

A post 2 meters tall at 6 meters casts the same image on your retina as a post 20 meters tall at 60 meters, or a toy 20 centimeters tall at 60 centimeters. The retinal image alone cannot say which. Any flat image is consistent with endlessly many three-dimensional scenes, and seeing depth means choosing among them. The visual system does so by relying on regularities that are usually true of the world: things are usually on the ground, objects of the same kind are usually the same size, near things hide far ones. Each depth cue is one such regularity. established

That is also why depth cues can be faked, by painters, by film-makers, and by the illusions in the next chapter.

Monocular cues

These work with one eye closed, and in a photograph or a painting; introductory texts list linear perspective, interposition (occlusion), relative size and closeness to the horizon among them[1], and optometry texts add aerial perspective and motion parallax[2]. In the figure, each one is a separate checkbox. established

  • Occlusion. A nearer object hides part of a farther one. With occlusion off, the posts are drawn see-through, so their overlaps no longer say which is in front.
  • Relative size. Objects you know to be the same size look smaller the farther away they are. The projected height of a post is f·H/Z: half the size at twice the distance.
  • Height in the visual field. Objects standing on the ground sit higher in the image, closer to the horizon, the farther away they are.
  • Linear perspective. Parallel lines, like the rails, converge toward a vanishing point on the horizon.
  • Texture gradient. Evenly spaced marks on the ground (here, lines every 2 meters) crowd together with distance.
  • Aerial perspective. Distant objects lose contrast through the haze of the air between you and them.
Question 1 Prediction: is any depth left with every cue off?#
Figure 1 The same scene with all six pictorial cues switched off: seven equal rectangles on one line, which could be at any distance. Drag sideways (or use the slider) to move your viewpoint; the posts separate by depth at once. Switch cues back on one at a time to see how much each one adds.#

The shift is easy to compute. If you move your head sideways by Δx, an object at distance Z shifts in the image by

shift = f · Δx / Z(1)

where f is the eye's (or camera's) focal length. Halve the distance and the shift doubles. The nearest post in the scene, at 4 meters, moves more than eleven times as far as the farthest, at 45. This is motion parallax, and it is one of the most powerful depth cues there is: it is why a passenger in a moving train sees nearby fences race past while distant hills hardly move. established

Two eyes, two views

Your eyes are a few centimeters apart: in adults the distance between the pupils averages about 63 millimeters, and most adults fall between 50 and 75 millimeters[4]. So each eye sees the world from a slightly different place. Hold a finger up close and look at it with each eye in turn: it jumps against the background. established

Question 2 Prediction: how does disparity change with distance?#

When you look at a point, both eyes turn so that its image falls on both foveas. Any other point at the same distance lands on matching spots in the two retinas. A point nearer or farther does not: its images are displaced in opposite directions in the two eyes. The angular difference is the retinal disparity (also called binocular disparity), and the brain's ability to read depth from it is stereopsis. For eyes a distance I apart fixating at distance df, an object straight ahead at do has disparity

δ = 2 arctan(I/2do) − 2 arctan(I/2df)(2)

and, for small depth differences,

δ ≈ I · Δd / d2(3)

Equation (2) is plain trigonometry; (3) holds when the depth difference Δd is small compared with the distance d. Disparity is positive, or crossed, for objects nearer than the fixation point, and negative, or uncrossed, for objects beyond it.

Figure 2 Two eyes, seen from above, fixating a cross at 1 m; a dot sits at 0.8 m. The lines of sight to the dot (dashed) cross nearer than the fixation lines, so its disparity is crossed. Drag the dot, or set the distances and your own pupil distance. On wide screens the boxes show what each eye sees: the dot on opposite sides of the cross. The readout gives the depth step that a 30-arcsecond disparity corresponds to at the fixation distance.#

The disparities involved are tiny. In a study of 188 biology students, Coutant and Westheimer found that 97% could see depth at a disparity of 2.3 minutes of arc or less, and at least 80% at 30 seconds of arc[5]. By equation (2), 30 arcseconds at one meter is a depth difference of about 2.3 millimeters. At arm's length, 60 centimeters, it is less than a millimeter. established

Disparity really is enough on its own. In 1960 Béla Julesz, writing in the Bell System Technical Journal, described pairs of computer-generated patterns of random dots in which a region was shifted sideways in one image relative to the other[6]. Either image alone shows nothing but dots. Seen one with each eye, the region floats out in depth. These random-dot stereograms showed that the brain can match the two eyes' views and find depth before it has recognized any shape at all, and they are the basis of the depth test on this site. established

Testing depth perception on a screen

A depth perception test has to remove every monocular cue, or it measures the wrong thing. Random-dot stereograms do that, but each eye must receive its own image. On an ordinary screen the cheapest way is an anaglyph: one eye's dots drawn in cyan, the other's in red, viewed through red–cyan glasses so that each eye sees only its own set.

Figure 3 Plate 1 of the red–cyan test: wear the glasses with red over the left eye and name the shape that floats in front of the dots. Use Left eye only or Right eye only to confirm that neither image contains a shape. The disparity shown for each plate assumes your viewing distance and a standard pixel size.#

Be clear about what such a test can measure. A screen's finest shift is one pixel. On this site's test, the last plate shifts each eye's dots by one 3-pixel cell, about 11 arcminutes of disparity at 50 centimeters on a standard screen: twenty times coarser than the 30 arcseconds most people can see. So the test shows whether stereopsis is working, not how good it is. Uncalibrated screens, glasses whose filters do not match the screen's colors and an unknown viewing distance all add error. Clinical stereoacuity tests use controlled printed plates or polarized displays and are interpreted by an eye care professional; this one is a demonstration. The full version is on the depth perception test page.

Not medical advice. If you could not see the shapes, the cause may be the glasses, the screen or the viewing conditions. If you are concerned about your depth perception or eye alignment, see an optometrist or ophthalmologist.

When the inference goes wrong

Every cue on this page is a bet about the world: that the posts are the same size, that they stand on the ground, that lines which converge in the image are parallel in reality. Usually the bets pay. When a picture is built to break them, you see a world that is not there. One famous explanation of the Müller-Lyer illusion says it is a depth cue applied to a flat drawing, and the next chapter tests that idea, along with three other illusions you can take apart.