What Can’t the Computer Do?

The Emperor's New Mind: Concerning Computers, Minds, and the Laws of Physics

by Roger Penrose
Oxford University Press, 466 pp., $24.95

The year 1989 marked a milestone in the history of artificial intelligence. A computer program, Deep Thought, defeated several chess grandmasters. The Russian grandmaster Garri Kasparov, it is true, defeated Deep Thought, but who can feel confident that in ten years’ time the then world champion will be able to defeat the best program? If a computer can succeed at the most beautiful and creative of games, what limits are there to the achievements of artificial intelligence? The question becomes more compelling if one accepts the current “strong AI” philosophy. According to this view, the human brain is only a large, if somewhat inaccurate, digital computer: consciousness is a necessary property of matter organized in such a way as to carry out complex computations: it follows, therefore, that when we construct computers as complex as the human brain, they too will be conscious. Computers will think as we do, and be aware of what they are doing.

Roger Penrose’s book was written to combat this strong AI philosophy. There are, he suggests, two other points of view open to you, even if you accept that the brain is the organ of thought, and hope for a scientific account of how it works. The first is that the brain is indeed a digital computer, but that it is conscious only because it is composed of living material. A digital computer made of transistors might perform identical computations, in identical ways, but it would not be conscious, because it was made of transistors and not of neurons. Penrose ascribes this view to John Searle, a philosopher who is critical of the strong AI view. I am not sure whether this correctly reflects Searle’s views: if not, no doubt he will find an opportunity to reply. In any case, Penrose does not find the argument convincing. For reasons I will give in a moment, I agree with him. Penrose’s own opposition to the strong AI view is differently based. The brain, he argues, is not a digital computer.

Before explaining why he thinks this, I must say a few words about the other line of argument, that computers are made of the wrong stuff to be conscious. As a geneticist, I am prejudiced against the idea that the peculiarities of living organisms arise because of the special nature of living material. This was a popular view when I was a boy, and was commonly used as a defense of religion against the inroads of science. I remember being told, by a schoolmaster who was also a parson, that scientists had shown that one could bring together all the chemical substances, carbon, oxygen, nitrogen, and so on, in the same proportions as are present in a seed, yet the seed would not grow into a plant, because it lacked the breath of life. Today we have a very satisfactory explanation of one of the two most fundamental properties of life—heredity—in chemical terms, so I hope that no schoolmaster is so foolish as to…

This is exclusive content for subscribers only.
Try two months of unlimited access to The New York Review for just $1 a month.

View Offer

Continue reading this article, and thousands more from our complete 55+ year archive, for the low introductory rate of just $1 a month.

If you are already a subscriber, please be sure you are logged in to your nybooks.com account.