WE LOOK AT THE WORLD from two slightly different vantage points, which correspond to the positions of our two eyes. These dual vantage points create tiny differences between the two eyes’ images that are proportional to the relative depths of objects in the field of view. The brain can measure those differences, and when it does so the result is stereovision, or stereopsis. To get an idea of this effect, extend one arm to point at a distant object. While keeping your arm extended, alternately open and close each eye. Notice how your finger shifts in relation to the object, illustrating the horizontal disparity between the eyes. Viewing devices that took advantage of stereopsis to create illusions of depth in images of natural scenes, architectural monuments and even pornography became immensely popular in Victorian drawing rooms. View-Master and Magic Eye are their familiar descendants available today. Brain Fusion A less commonly appreciated fact about stereovision is that even though we see two images of an object—one through each eye—we perceive only one object. (In similar fashion, if you touch a single banana with your two hands, you feel one banana, not two.) Thus, the images of the two eyes must be “fused” somewhere in the brain to give rise to a unitary item of perception, or a percept. But we can ask the questions, What if the eyes look at completely dissimilar things? Would we perceive a blend? Try the following experiment. Get low-power reading glasses, such as you can find in any drugstore. Affix two colored filters, one bright red and one bright green, to the front of each lens. Put the glasses on. If you now look at, say, a white object or surface, what do you see? If you close one eye or the other, you see red or green as expected. But what if you open both? Do the two colors harmonize and blend in your brain to produce yellow as they would if blended optically? (As any preschooler knows, red and green make brown if you mix pigments like tempera paints. But if you mix lights by projecting them onto a screen, red and green produce yellow.) The surprising answer is that you see only one thing at a time. The object appears alternately red and then green. The eyes seem to take turns politely, as if to avoid conflict. This phenomenon is called binocular rivalry, and the effect is similar to what you see in the Necker cube (a). To the viewer, it may seem as though these dynamic perceptual experiences arise because the object is itself changing. Yet the stimulus is perfectly stable, and it is instead the pattern of brain activity that is changing during viewing and producing the perceptual alterations or the illusion of an unstable object. We can use rivalry as a powerful tool to explore the more general question of how the brain resolves perceptual conflicts. Let’s try another experiment. Instead of two different colors, what if you give the eyes two sets of stripes that are perpendicular to each other? Would they produce a grid? Or do they clash? The answer is that sometimes you see them alternate—but equally as often you see a mosaic of patches, with sections of both eyes’ images interleaved (b, on preceding page). No grid. Theoretically, you could do this experiment by putting vertical stripes for the right eye and horizontal for the left in a stereo viewer. But if you do not have one, you can create what we call the poor man’s version (c, on preceding page). Just prop up a vertical partition, like a manila folder, right at the boundary between two images corresponding to left and right eyes. Put your nose on the partition so the left eye looks exclusively at one image and the right eye at the other. You will see either the stripes alternating or a fluctuating mosaic but never a grid. With practice you can dispense with the partition and just learn to “free fuse” the two images by converging or diverging your eyes. It helps if you initially look at a pencil tip halfway between the images and your face. Once you have learned this trick, you can try a number of new things. We know, for example, that different areas of the brain are involved in processing color and form of visual images. So we can ask, Does the rivalry occur separately for these two, or do they always happen together? What if you looked at the left eye’s stripes through a red filter and the right eye’s through a green one? There will now be both rivalry of color and rivalry of form. Can these two rivalries come about independently, so that the left eye’s color goes with the right eye’s stripes, or do they always “rival” synchronously? The short answer is that they do so together. Putting it crudely, the rivalry is between the eyes themselves rather than in processing the colors or shapes. Complete the Picture But that is not always true. Consider the curious display in d. Each eye’s picture is a composite of a monkey’s face and foliage. Intriguingly, if the brain fuses the images, it has a strong tendency to complete either the monkey or the foliage—even though doing so requires assembling fragments from two different eyes to complete the patterns. In this case, the brain is picking bits from each eye that make “sense” as a holistic pattern when combined correctly. Let’s return to stereopsis, the computation of relative depth from images in the two eyes that are slightly different because the eyes are separated horizontally in the skull. Here both image fusion and stereo depth occur instead of rivalry. It is quite remarkable that people wandered our planet for millennia without recognizing stereopsis (probably assuming that the benefit of two eyes is that if you lose one you have a spare). Leonardo da Vinci pointed out that this information existed 500 years ago; the fact that the brain actually uses it was discovered by Victorian physicist Charles Wheatstone. You can create an example of Wheatstone’s discovery by viewing a drawing of a bucket shape as seen from the top. When you fuse the two eyes’ pictures (using free fusion or the partition card), a gray disk jumps out at you—as if suspended mysteriously in thin air—from the plane of the outer circle. But do you need fusion for stereopsis to occur? This may seem like a trick question, because one would think so intuitively, but that intuition is wrong. Three decades ago Anne Treisman of Princeton University, Lloyd Kaufman of New York University and one of us (Ramachandran) independently showed that, paradoxically, rivalry can coexist with stereopsis. To understand this phenomenon, look at the stereogram shown in e. It has two patches of stripes shifted horizontally in opposite directions relative to the outer circles. When the brain fuses these circles, something extraordinary happens. You will see the entire patch floating out in front—yet only one patch at a time, because the stripes themselves are orthogonal. In other words, the brain extracts the stereo signal from the patches as a whole—interpreting the individual chunks as blobs—yet those patches themselves are seen to rival. The information about the location of the patches on the retina is extracted by the brain and produces stereopsis, even though only one eye’s image is visible at a time. It is as if information from an invisible image can nonetheless drive stereopsis. Such “form rivalry” occurs in a different brain area from stereopsis, so the two can coexist in harmony. The correlation between them in normal binocular vision is coincidental, not obligatory. This discovery that certain visual information can be processed unconsciously in a parallel brain pathway reminds us of the enigmatic neurological syndrome of blindsight. A patient with damage to the visual cortex is completely blind. He cannot consciously perceive a light spot. But he can reach out and touch it using a parallel pathway that bypasses the visual cortex (which you need for conscious awareness) and projects straight to brain centers that are on a kind of autopilot to guide your hand. A similar experiment could, in theory, be done for binocular rivalry. When one eye’s image is suppressed entirely during rivalry, can you nonetheless reach out and touch a spot presented to that eye, even though that spot, for the suppressed eye, is invisible? The phenomenon of rivalry is a striking example of how you can use relatively simple experiments to gain deep insights into visual processing.
To get an idea of this effect, extend one arm to point at a distant object. While keeping your arm extended, alternately open and close each eye. Notice how your finger shifts in relation to the object, illustrating the horizontal disparity between the eyes.
Viewing devices that took advantage of stereopsis to create illusions of depth in images of natural scenes, architectural monuments and even pornography became immensely popular in Victorian drawing rooms. View-Master and Magic Eye are their familiar descendants available today.
A less commonly appreciated fact about stereovision is that even though we see two images of an object—one through each eye—we perceive only one object. (In similar fashion, if you touch a single banana with your two hands, you feel one banana, not two.) Thus, the images of the two eyes must be “fused” somewhere in the brain to give rise to a unitary item of perception, or a percept. But we can ask the questions, What if the eyes look at completely dissimilar things? Would we perceive a blend?
Try the following experiment. Get low-power reading glasses, such as you can find in any drugstore. Affix two colored filters, one bright red and one bright green, to the front of each lens. Put the glasses on. If you now look at, say, a white object or surface, what do you see? If you close one eye or the other, you see red or green as expected. But what if you open both? Do the two colors harmonize and blend in your brain to produce yellow as they would if blended optically? (As any preschooler knows, red and green make brown if you mix pigments like tempera paints. But if you mix lights by projecting them onto a screen, red and green produce yellow.)
The surprising answer is that you see only one thing at a time. The object appears alternately red and then green. The eyes seem to take turns politely, as if to avoid conflict. This phenomenon is called binocular rivalry, and the effect is similar to what you see in the Necker cube (a). To the viewer, it may seem as though these dynamic perceptual experiences arise because the object is itself changing. Yet the stimulus is perfectly stable, and it is instead the pattern of brain activity that is changing during viewing and producing the perceptual alterations or the illusion of an unstable object.
We can use rivalry as a powerful tool to explore the more general question of how the brain resolves perceptual conflicts. Let’s try another experiment. Instead of two different colors, what if you give the eyes two sets of stripes that are perpendicular to each other? Would they produce a grid? Or do they clash? The answer is that sometimes you see them alternate—but equally as often you see a mosaic of patches, with sections of both eyes’ images interleaved (b, on preceding page). No grid.
Theoretically, you could do this experiment by putting vertical stripes for the right eye and horizontal for the left in a stereo viewer. But if you do not have one, you can create what we call the poor man’s version (c, on preceding page). Just prop up a vertical partition, like a manila folder, right at the boundary between two images corresponding to left and right eyes. Put your nose on the partition so the left eye looks exclusively at one image and the right eye at the other. You will see either the stripes alternating or a fluctuating mosaic but never a grid. With practice you can dispense with the partition and just learn to “free fuse” the two images by converging or diverging your eyes. It helps if you initially look at a pencil tip halfway between the images and your face.
Once you have learned this trick, you can try a number of new things. We know, for example, that different areas of the brain are involved in processing color and form of visual images. So we can ask, Does the rivalry occur separately for these two, or do they always happen together? What if you looked at the left eye’s stripes through a red filter and the right eye’s through a green one? There will now be both rivalry of color and rivalry of form. Can these two rivalries come about independently, so that the left eye’s color goes with the right eye’s stripes, or do they always “rival” synchronously? The short answer is that they do so together. Putting it crudely, the rivalry is between the eyes themselves rather than in processing the colors or shapes.
But that is not always true. Consider the curious display in d. Each eye’s picture is a composite of a monkey’s face and foliage. Intriguingly, if the brain fuses the images, it has a strong tendency to complete either the monkey or the foliage—even though doing so requires assembling fragments from two different eyes to complete the patterns. In this case, the brain is picking bits from each eye that make “sense” as a holistic pattern when combined correctly.
Let’s return to stereopsis, the computation of relative depth from images in the two eyes that are slightly different because the eyes are separated horizontally in the skull. Here both image fusion and stereo depth occur instead of rivalry. It is quite remarkable that people wandered our planet for millennia without recognizing stereopsis (probably assuming that the benefit of two eyes is that if you lose one you have a spare). Leonardo da Vinci pointed out that this information existed 500 years ago; the fact that the brain actually uses it was discovered by Victorian physicist Charles Wheatstone. You can create an example of Wheatstone’s discovery by viewing a drawing of a bucket shape as seen from the top. When you fuse the two eyes’ pictures (using free fusion or the partition card), a gray disk jumps out at you—as if suspended mysteriously in thin air—from the plane of the outer circle.
But do you need fusion for stereopsis to occur? This may seem like a trick question, because one would think so intuitively, but that intuition is wrong. Three decades ago Anne Treisman of Princeton University, Lloyd Kaufman of New York University and one of us (Ramachandran) independently showed that, paradoxically, rivalry can coexist with stereopsis.
To understand this phenomenon, look at the stereogram shown in e. It has two patches of stripes shifted horizontally in opposite directions relative to the outer circles. When the brain fuses these circles, something extraordinary happens. You will see the entire patch floating out in front—yet only one patch at a time, because the stripes themselves are orthogonal. In other words, the brain extracts the stereo signal from the patches as a whole—interpreting the individual chunks as blobs—yet those patches themselves are seen to rival.
The information about the location of the patches on the retina is extracted by the brain and produces stereopsis, even though only one eye’s image is visible at a time. It is as if information from an invisible image can nonetheless drive stereopsis.
Such “form rivalry” occurs in a different brain area from stereopsis, so the two can coexist in harmony. The correlation between them in normal binocular vision is coincidental, not obligatory. This discovery that certain visual information can be processed unconsciously in a parallel brain pathway reminds us of the enigmatic neurological syndrome of blindsight. A patient with damage to the visual cortex is completely blind. He cannot consciously perceive a light spot. But he can reach out and touch it using a parallel pathway that bypasses the visual cortex (which you need for conscious awareness) and projects straight to brain centers that are on a kind of autopilot to guide your hand.
A similar experiment could, in theory, be done for binocular rivalry. When one eye’s image is suppressed entirely during rivalry, can you nonetheless reach out and touch a spot presented to that eye, even though that spot, for the suppressed eye, is invisible?
The phenomenon of rivalry is a striking example of how you can use relatively simple experiments to gain deep insights into visual processing.
