Inner Vision: An Exploration of Art and the Brain
by Semir Zeki
Oxford University Press, 224 pp., $35.00
Visual Intelligence: How We Create What We See
by Donald D. Hoffman
Norton, 294 pp., $17.95 (paper)
By the end of the nineteenth century neurologists were convinced that seeing and understanding were two distinct, anatomically separate brain functions; seeing was passive and understanding active. The evidence seemed clear: patients with damage in one part of the brain became blind, whereas patients with damage in another part of the brain were able to see but could not understand what they were seeing. They could not recognize objects. The German neurologist Hermann Munk called this second condition “mind-blindness” (Seelenblindheit) and today it is known as “agnosia,” a term we owe to Sigmund Freud. These neurological findings led to the widely accepted idea that the eye was like a camera: the eye took photographs and transmitted them to the visual center of the brain; the photographs were then interpreted and understood by other cortical centers. When the cortical centers were damaged and could no longer interpret them, the result was agnosia.
During the past thirty-odd years, however, neurophysiological and clinical studies, as well as attempts to simulate animal and human vision on computers, have given us a considerably more sophisticated view of the nature of perception. Since the world we see is always changing and the retina receives a constant flow of different kinds of visual information, the brain must be able to select visual properties of objects and surfaces in order to give them coherence. In acquiring this ability, the brain has developed specialized functions for the analysis of different properties, such as color, shape, and movement. It creates our visual worlds, including the colors of the rainbow, the illusion of motion on the movie screen, and the perception of depth in a flat painting.
Semir Zeki, a professor of neurobiology at University College, London, has made some of the more important discoveries that have radically changed our view of how we see and understand the world around us. His book A Vision of the Brain, published in 1993, has become a classic introduction to vision and the brain sciences, showing how recent neurophysiological studies support the view that our visual worlds are a product of complex processes rather than a simple reflection of the world around us. Though the different colors we see are determined by the different frequencies of light reflected off surfaces, the particular sensation of, for example, the color red is a product of our brains. The perception of color is one of the ways the brain makes us aware of the physical characteristics of the external world, in this case various frequencies of light. The universe, as Isaac Newton noted long ago, is colorless: “Rays, to speak properly, have no Colour. In them there is nothing else than a certain power and disposition to stir up a sensation of this Colour or that.”
In Inner Vision, Zeki is less concerned with what neurophysiology can tell us about art than what art can tell us about neurophysiology. Painters have always been dependent on the functional organization of the visual brain, without being aware of that dependence. The portrait painter, Zeki points out, relies on the specialized areas in the brain that allow us to see and identify faces. So, too, attempts of Fauve painters such as Matisse or Derain to liberate colors from form could only be achieved because color and form are separate and independent brain functions. And the Cubists, Zeki writes, were able to make explicit our simultaneous awareness of multiple perspectives because specific brain mechanisms can analyze the two-dimensional images on our retinas as three-dimensional objects.
Zeki’s new book fortuitously appears at the same time as Visual Intelligence by Donald Hoffman, professor of the cognitive sciences at the University of California (Irvine), which gives a remarkably clear summary of what we know, and don’t know, about the way the brain creates what we see. Hoffman has done important research suggesting that in order to create visual perceptions the brain must be using a set of rules comparable to the rules of transformational grammar that Noam Chomsky has found to be universal characteristics of language. His book describes implicit rules that make it possible for us to see lines, depth, colors, form, and motion. Taken together, the books by Zeki and Hoffman give the most perceptive general account we have of the workings of human vision.
In A Vision of the Brain, Zeki recalled that in 1888 a Swiss ophthalmologist, Louis Verrey, published a study of a sixty-year-old woman who had a stroke affecting her left hemisphere. When she looked straight ahead, everything to her right (her “right visual field”) appeared gray. The post-mortem examination of her brain revealed localized brain damage and Verrey therefore concluded that there was a subdivision of the visual cortex that was specifically devoted to color. “The implications of Verrey’s conclusion,” Zeki wrote,
were momentous, indeed so momentous that even he failed to see them. For, if color vision could be specifically and separately compromised, this would suggest that color is separately represented in the brain. From this it would follow that functional specialization is a much more widespread phenomenon, extending to the submodalities of a modality, in this case the sensation of color within vision. It would also follow that the cerebral processes involved in vision are not unitary, as our unitary experience of the visual world might suggest. Many profound questions about the brain and about vision would have followed. It is no wonder that this finding was too revolutionary to many.
The full implications of Verrey’s work became clear in 1973 when Zeki discovered the area in the rhesus monkey’s brain that is specialized for seeing colors. When this area is destroyed in the monkey or human brain, neither the monkey nor the human can see colors. At the time of Zeki’s discovery neurophysiologists were beginning to establish the outlines of a new view of the visual cortex, suggesting that there are specific functional units for seeing form, motion, and colors. Instead of interpreting “pictures” in the head, the brain is performing a variety of highly specialized visual subtasks. It is creating pictures by combining the operations of the different specialized regions into a unified visual image, though just how the brain does this is still not understood. The brain, Zeki writes, “is no mere passive chronicler of the external physical reality but an active participant in generating the visual image, according to its own rules and programs.”
Hoffman’s approach is explicit in comparing these rules and programs to Chomsky’s approach to language. “The rules of universal grammar,” Hoffman writes,
allow a child to acquire the specific rules of grammar for one or more specific languages…. Similarly, the rules of universal vision allow a child to acquire specific rules for constructing visual scenes. These specific rules are at work when the child, having learned to see, looks upon and understands specific visual scenes.
Consider the well-known image in which figure and ground can be reversed—either one sees two faces or one sees a goblet. The two possibilities are governed, according to Hoffman, by the following rule: the points of greatest curvature in the concave region of a curve determine what we take to be the “parts” of the image. The point of greatest curvature refers to the region “inside” the curve where the angle of change is the greatest. (For example, in a curve that shows a sudden drop in the Dow Jones average followed by a complete recovery one day later, the point at which the Dow Jones is at its lowest is where the angle of the curve is the greatest, representing as it does the Dow Jones going from high to low and then back to high.) Thus if we see the goblet as a “figure,” the points of greatest curvature “inside” the curve create the lip, the bowl, the stem, and the base; whereas if we see the two faces as a “figure,” the points of greatest curvature “inside” the curve are the forehead, nose, lips, and chin (see illustration on page 62). What is the “inside” of the curve for the goblet is the “outside” of the curve for the face. Hoffman writes that we still do not understand why, at any given moment, the viewer hits on one set of curves rather than another.
What has become clear, however, is that the brain creates our visual world by connecting and then transforming stimuli. For example, motion pictures give us a sense of continual movement by means of a series of static images presented in rapid succession. Our visual experience is not of one static image followed by another; instead we see motion because the brain—specifically that part of the brain that is specialized in the analysis of visual motion—unconsciously relates one static image to the next. We comprehend motion when the static images are presented at the rate of twenty-four frames per second. At any slower motion we see only a succession of static frames, which suggests that the brain establishes visual motion from stimuli occurring about twenty-fourths of a second apart. Without this activity of connecting, we would merely perceive a sequence of unrelated stimuli from moment to unrelated moment.
The catastrophic consequence of the inability to see movement is described in a famous case, reported in Hoffman’s book, of a woman who, following her recovery from a stroke, reported that she could no longer see things move. She had great difficulty pouring tea from a teapot because the fluid in the pot appeared to her to be frozen:
In addition, she could not stop pouring at the right time since she was unable to perceive the movement in the cup (or a pot) when the fluid rose. Furthermore the patient complained of difficulties in following a dialogue because she could not see the movements of the face and, especially, the mouth of the speaker. In a room where more than two other people were walking she felt very insecure and unwell, and usually left the room immediately, because “people were suddenly here and there but I have not seen them moving.” [She had the] same problem but to an even more marked extent in crowded streets or places, which she therefore avoided as much as possible. She could not cross the street because of her inability to judge the speed of a car, but she could identify the car itself without difficulty. “When I’m looking at the car first, it seems far away. But then, when I want to cross the road, suddenly the car is very near.”
The failure, then, to see or judge motion is a failure to establish relations that depend, in Hoffman’s view, on the rules that are part of a universal grammar of vision:
No one teaches you these rules. Instead, you acquire them early in life in a genetically predetermined sequence that requires, for its unfolding, visual experience…. And just as an adult, using rules of grammar, can understand countless sentences (in principle, if not in practice), so also an adult, using rules of vision, can understand countless images (again, in principle if not in practice).