The New York Review of Books

To the Editors:

In his review of Penrose’s The Emperor’s New Mind [NYR, March 15], John Maynard Smith expresses some doubt about whether the views he attributes to me are in fact mine. His doubts are justified. I do not hold the views that a computer “would not be conscious, because it was made of transistors and not of neurons.” That is not my view at all. My position rather, is this: I take it as a fact that certain quite specific, though still largely unknown, neurobiological processes in human and some animal brains, cause consciousness. But from the fact that brains cause consciousness we can derive trivially that any other system capable of causing consciousness would have to have the relevant causal powers at least equivalent to brains. If brains do it causally then any other system that does it causally will have to share with brains the power to do it causally. I hope that sounds tautological, because it is. Some other system might use a different chemistry, a different medium altogether; but any such medium has to be able to do what brains do. (Compare: airplanes don’t have to be made of feathers in order to fly, but they do have to share with birds the causal capacity to overcome the force of gravity in the earth’s atmosphere.)

This obvious result becomes interesting when tied to another result, equally obvious. Just implementing a computer program is not sufficient by itself to guarantee the presence of mental contents, conscious or otherwise, because the program is defined entirely in terms of abstract symbol manipulation, and such abstract syntactical objects do not guarantee the presence of mental phenomena with semantic content. Syntax is not by itself the same as, nor is it sufficient for, semantics. This is shown by my Chinese Room Argument. A “computer,” me for example, might follow the rules of a program for “understanding” questions in Chinese and giving the right answers in Chinese and still not understand a word of Chinese. Now these two results, each in its way obvious, lead to some other interesting conclusions when they are conjoined. First, it follows that the way that brains produce conscious mental contents cannot be solely in virtue of implementing a formal computer program. And second, any artifact could not have specific mental contents solely in virtue of implementing a program but would have to have the relevant causal powers equivalent to the brain. (I am summarizing here a much longer argument, for more details see, for example, my Mind, Brains and Science, Harvard University Press, 1984.) The result is not that “a computer can’t be conscious,” nor is it that “only systems made of neurons can be conscious,” but rather, implementing a computer program is not by itself sufficient to guarantee the presence of mental contents, conscious or otherwise.

I think Maynard Smith is prevented from seeing these points, because he holds a set of very deep but unstated assumptions about the relation of the mind to the body, about the nature of computation and about the nature of consciousness. He is certainly not responsible for these views. They are part of our contemporary culture. Nonetheless, they are mistaken and I would like to state—all too briefly—what some of them are and why they are mistaken. For the sake of brevity, I will confine myself to half a dozen of these basic assumptions.

1. He treats the question, “Is the brain a digital computer?” as if it were is simple factual question, as if we might simply discover that the processes in the brain are computational in the way that we have discovered that the heart is a pump or that green plants do photosynthesis. He even discusses at some length the question as to which functions of the brain might be performed by analog computation, which by digital computation.

But I hope he would agree with me on reflection that the question is more complex than his discussion implies. The reason can be stated succinctly: In the standard textbook definition “digital computation” does not name a set of physical processes in virtue of their physical features, but names a set of abstract formal symbolic processes which can go on in an indefinitely large range of physical media. But now we immediately face a difficulty. The question, “Is this a digital computer?” is not like the question, “Is this a pump?” or “Is this process photosynthesis?” but more like the question, “Is this a symbol?” or “Is this set of objects a set of symbols?” But being a symbol is not just a matter of physical properties; rather a physical object is a symbol only insofar as it is used or could be used by some agent as a symbol. Similarly, on the definition of computation as symbol manipulation, being a digital computer is not just a matter of physical properties; rather something is a digital computer if it can be used to compute with or can be described in computational terms. But then the problem is that anything that meets certain minimal formal conditions can be described as a digital computer, because just about anything can be assigned a symbolic interpretation. We can describe its operation using the famous 0’s and 1’s. Thus, molecules, solar systems, and beer cans are all in a trivial sense digital computers, because they can all be described as implementations of computer programs. Oddly enough, a similar point about analog computers is already implicit in Maynard Smith’s discussion. His examples of analog computers are match sticks and bits of string. But if match sticks and bits of string can be analog computers, then just about anything can be an analog computer. Similarly, just about anything can be a digital computer.

But this gives us a dilemma where our original question is concerned. Are brains digital computers? If this asks whether brains can be described in computational terms, the answer is in a trivial sense yes, because just about anything at all can be described in computational terms. But that was not the question we wanted to ask: We wanted to ask whether brains are somehow intrinsically digital computers, and that question has not so far been given a clear sense.

I do not say that we could not succeed in giving it a clear sense, but rather that we have not so far done it. This point is different form the Chinese Room Argument, but is a natural extension of it: the Chinese Room Argument showed that semantics is not intrinsic to syntax, the point I am making now is that syntax is not intrinsic to physics.

Is there any way out of this dilemma? Yes there is and it is a route that is standardly taken in artificial intelligence and cognitive science as well as in Maynard Smith’s review. But it is out of the frying pan and into the fire.

1. The standard way to avoid the dilemma is to describe the brain as if its processes were used for computation, in the same sense that, for example, regular commercial computers are used for computation. This works fine for commercial computers because such computers are precisely designed, programmed, and bought to be used for computation by some outside agent; but to treat the brain this way is to commit a homunculus fallacy. (The homunculus fallacy is the fallacy of explaining our mental processes by tacitly postulating a “little man,” a homunculus, in our heads who is having those thought processes for us.) On several occasions (e.g., on p. 22) Maynard Smith speaks of “representations” as being “used” for computation. But think about that for a minute. Who is doing the using? And remember, something is a representation only to the extent that some agent uses it as a representation. So if we even consider the possibility that, as he puts it, “the representation is used in analog computing” we have to attribute some mental capacity to the brain that stands outside the representation and could use it as a representation. This is the familiar form of the homunculus fallacy.

2. The combination of 1 and 2 leads him to misdescribe the actual biology of cognition.

If you think that there is a little man in the head using the brain to compute you will say things like the following: “But the problem of vision is to explain how this [2-D retinal] image is translated into ‘there is a car approaching me on the wrong side of the road.’ ” I think if Maynard Smith reflects on this he will agree with me that as it stands this account makes little biological sense. Biologically speaking what actually happens is something like this: A series of photons strike the photoreceptor cells in my retina. This signal is then processed through four other layers of the retina and passes through the optic nerve to the lateral geniculate nucleus. From the LGN the signal goes to the striate cortex, zone 17, and then through the rest of the visual cortex, through zones 18 and 19. Eventually this complex electro-chemical process causes a concrete conscious visual experience. As a piece of biology, the whole process is as specific as, say, digestion, and like digestion it is a specific causal chain of events that results in concrete biological events, in this case it ends in a concrete mental (hence physical) event of me seeing this very scene. Someone, I or someone else, might describe the content of the visual experience as “There is a car approaching me on the wrong side of the road”; but the concrete biological visual reality is not that of a bunch of words, it is an actual conscious experience. And there is literally no “translation” going on, nor is there literally any homunculus computing over the visual image.

It is always possible to describe vision or other mental processes in abstract computational information processing terms, as one can describe any process in these terms, biological or otherwise. Digestion can also be described as a computational information processing sequence, and there is nothing harmful about these descriptions provided you don’t confuse the computational model with the real thing. Nobody supposes that the question, “Is the stomach a digital computer?” is the right question to ask, even though the stomach can be described computationally and thus can be simulated on a digital computer. A computational model of a vision will indeed “translate” information about a two-dimensional visual array into the sentence, “There is a car approaching me on the wrong side of the road.” But that gives us a model of a visual process, not a visual process.

No one confuses model and reality where digestion is concerned; why does anyone make the confusion where consciousness is concerned? Part of the reason is this:

1. Maynard Smith, though not a dualist, is still making use of certain dualistic categories. He has difficulty in seeing how the subjective inner mental state of consciousness can be part of the ordinary biological world of digestion, photosynthesis, the secretion of bile, and mitosis. It is, I believe, his difficulty in seeing that consciousness is an ordinary higher level feature of the brain (in the same sense that the solidity of this table is a higher level feature of the table), that leads him to say such things as “What I find most puzzling about Penrose’s position is that he wants consciousness to ‘do something.’ He writes as if consciousness were an additional cause of thought, or of behavior, over and above the physical events in the brain.” But how are we to take this puzzlement except as an expression of the traditional dualistic assumption that “consciousness” and “physical events” name mutually exclusive categories? Once you see that consciousness, i.e. the subjective, inner mental experience of consciousness, is indeed a higher level physical feature of the brain—and only the dualist assumption that physical and “mental” are mutually exclusive prevents us from seeing that—then there is no philosophical puzzlement about how consciousness can function causally. Of course there are plenty of factual problems about how it works. We have only scratched the surface.