[review of] Roger Penrose, *The Emperor's New Mind: Concerning Computers, Minds, and the
Laws of Physics*, Oxford Univ. Press, 1989.

The idea that a computer could be conscious--or equivalently, that human consciousness is the effect of some complex computation mechanically performed by our brains--strikes some scientists and philosophers as a beautiful idea. They find it initially surprising and unsettling, as all beautiful ideas are, but the inevitable culmination of the scientific advances that have gradually demystified and unified the material world. The ideologues of Artificial Intelligence (AI) have been its most articulate supporters. To others, this idea is deeply repellent: philistine, reductionistic (in some bad sense), as incredible as it is offensive. John Searle's attack on "strong AI" is the best known expression of this view, but others in the same camp, liking Searle's destination better than his route, would dearly love to see a principled, scientific argument showing that strong AI is impossible. Roger Penrose has set out to provide just such an argument.

It is a huge project. In order to build his case, Penrose must lead the reader through detailed
discussions of many topics in mathematics (Turing machines and computability theory, complex
numbers, the Mandelbrot set, Gödel's theorem, recursive function theory, complexity theory,
Platonism versus intuitionism), classical Einsteinian physics and quantum physics, cosmology,
and, of course, neuroscience. Most of these topics have been given excellent popular
presentations in recent years--in Hofstadter's *Gödel Escher Bach* (Basic Books, 1979), Hawking's
*A Brief History of Time* (Bantam, 1988), Gleick's *Chaos: Making a New Science* (Penguin,
1987)--but Penrose believes that he must go over this material again in his own way, digging
deeper, explaining in more detail. The result is bracing reading, to say the least, and the topics for
hundreds of pages on end apparently have nothing to do with the mind at all.

The inevitable first impression, then, is that the book is the ultimate academic shaggy dog story, a tale whose fascinating digressions outweigh the punch line by a large factor. Why does Penrose do it? Is there no swifter, more accessible route to his conclusion? No. Penrose sees that he has no hope of overthrowing the case for strong AI unless he can dislodge one of the most imperturbable objects in the intellectual universe: something I will call the Cathedral of Science.

The Cathedral of Science is the highly structured, beautifully articulated amalgam of "what
everyone should know" about science, crowned by the inscrutable but talismanic formula,
"e=mc2.". Its facade, visible to the general public, is popular lore: familiar and decorative items of
information and misinformation about the Galilean physics of everyday objects, cartoon-style
renderings of black holes and language-using chimps, and pockmarked with such tidbits as "you
only use five percent of your brain" and "no two snowflakes are alike." Items in this layer are
easily replaced or swept away, but underneath it lies the scientists' (and philosophers') much
denser version of the same material, created largely of the remembered oversimplifications of
university-level textbooks, supplemented by *New Scientist* and *Scientific American* articles, and
such other high-quality interdisciplinary communications as the books just mentioned. This
material forms the communally shared understanding on which everyone relies while working on
their more particular specialities. Aside from a few brilliant polymaths, the neuroanatomist, the
biochemist, the experimental psychologist and the philosopher of mind have roughly the same
workaday understanding of quantum mechanics, entropy, and computability, for instance, and this
understanding gives them sufficient reason to believe that they needn't understand these topics any
better in order to do their work. The Cathedral's architecture is the familiar hierarchy of
mechanistic materialism: living bodies are self-preserving, self-replicating machines made out of
cells made out of molecules made out of atoms--with some weird quantum physics isolated (one
hopes) in the cellar. No church has ever enjoyed a more entrenched or authoritative orthodoxy, an
empire that expands with daily discoveries and protects itself from swift change by the distributed,
mutual myopia of its adherents. Its heresies (ESP, creationism, vitalism) are readily identified and
deplored in unison; its conservatism is hailed by almost all who participate in it, and for good
reason.

It is Penrose's immense task to restructure our vision of the Cathedral of Science, shaking our conviction that it is largely settled and safe and familiar (except, of course, for that baffling business about quantum physics). His task is made all the more intricate by his recognition that most of the Cathedral is sound. He is a revolutionary, but no bomb-throwing nihilist. Like Archimedes, he needs a place to stand if he is to move the world, so he introduces a new taxonomy of theories in science, SUPERB, USEFUL and TENTATIVE, to distinguish what is inviolable and must somehow survive any revolution, from what might be replaced or abandoned. Euclidean geometry, Galilean dynamics, Maxwell's equations, Einstein's special and general relativity theories, quantum physics and quantum electrodynamics are all SUPERB, but even these must be put into a new alignment if we follow Penrose.

Briefly, here is the path of Penrose's proposed revolution. If the brain is a computer, its powers
are circumscribed by the limits on all computation uncovered by Turing and Gödel. Turing
showed that each possible mechanical computation can be precisely specified by a recipe
consisting of a sequence of dead-simple mechanical steps. Such a recipe is called an algorithm; all
computer programmes are algorithms. Gödel's Theorem showed that no algorithm for proving
mathematical truths can prove them all. Doesn't Gödel's Theorem establish that there are tasks
"we" (mathematicians, in any event) can perform that are beyond the capabilities of any machine?
The idea that the human race can be saved from machinehood by riding on the coattails of those
clever enough to understand Gödel's Theorem is well-explored territory, and the received wisdom
is that all the previous arguments for this conclusion have been roundly defeated, so if Penrose is
to get his needed premise here, he must find a new wrinkle. The standard Cathedral vision is that
Gödel's Theorem proves that there is just some single arcane truth of number theory (a machine's
Gödel sentence) that is beyond all mechanical computation (by that machine), and Penrose's
detailed exposition of the wealth of non-computable but knowable results replaces that vision with
an appreciation of the depth and importance of the realm of non-computable mathematics,
certainly a domain that is eminently accessible to human mathematicians relying on "insight".
Moreover, the results of complexity theory show that there are many officially computable results
that are not *practically* computable--the algorithms that are guaranteed to yield the answer would
take billions of years to run on the fastest conceivable computers. How, then, do "we" arrive at
solutions to these problems? Penrose proposes that there is a "*theoretical* possibility that a
quantum physical device may be able to improve on a Turing machine." (p.146)

This leads then to a solid review of classical (non-quantum, but relativistic) physics, packed with novel perspectives and designed to impress us that "we should not be too complacent that the pictures that we have formed at any one time are not to be overturned by some later and deeper view." (p.217) With our minds thus stretched open, we plunge into "quantum magic and quantum mystery," and are led yet one more time through the two-slit experiment, Heisenberg's Uncertainty Principle, the collapse of the wavefunction, the Einstein Podolsky Rosen paradox and Schrödinger's notorious cat--in more detail than I have heretofore encountered in a popular book. Here the upshot is more radical: Penrose doubts that the puzzles of quantum theory and its relation to classical theory will succumb to any tidy, local resolution, and, like Einstein, he resists the standard "anti-realist" interpretations favored by most physicists. After a further chapter laying groundwork in cosmology on the flow of time and the curious status of the second law of thermodynamics, we are ready for the suggestion that if a unified theory is to be found, it will have to be a theory of "quantum gravity," requiring "a change in the very framework of the quantum theory." (p.348) At this time Penrose can present only speculations about the "germ" of such a theory, which is not yet, in his own terms, even TENTATIVE, so he has to settle for some gestures in the direction he feels the revolution will take.

Finally, then, what does all this have to do with minds and brains? He returns to the topics of the early chapters, and resumes the argument that mathematical insight (in particular) is non-algorithmic. Here is where consciousness comes in. The function of consciousness, in Penrose's view, is to leapfrog the limits of (practical) computability by conjuring up appropriate judgments in circumstances in which "enough information is in principle available for the relevant judgment to be made, but the process of formulating the appropriate judgment, by extracting what is needed from the morass of data, may be something for which no clear algorithmic process exists--or even where there is one, it may not be a practical one." (p.412) The way a "quantum computer" would achieve this apparent magic would be by being a sort of super-parallel computer, using superposition of computational states to perform a near-instantaneous global search through an otherwise untraversable space of possibilities, with the solution being output by the collapse of the wavefunction. This would not be old-fashioned Cartesian dualism, but radically new-fashioned (revolutionized) materialism. Several features of what is currently known or believed about connectivity of neural nets in the human brain suggest to him that the brain could in principle be such a quantum computer.

One of the defining doctrines of strong AI is the possibility in principle of teleportation--transporting a *person* from A to B by transmitting a complete, atom-for-atom *description* of the
person's body (and brain) and using the description to construct a duplicate at the destination. Is
teleportation murder-and-artifice or a means of transportation? Popular science fiction has for
years softened us up for the latter vision, but Penrose is among those who find this idea simply
incredible, and one of the cardinal virtues of the quantum computer idea, in his eyes, is that it
would rule out perfect duplication of the total quantum state of a brain on what he argues are
relatively secure quantum-physical principles.

Where, though, would Penrose have quantum physics draw the line? In principle, could a geranium in a pot be teleported? (When as a child I first heard of "sending flowers by wire" I assumed that they were teleported, and was deeply disappointed to learn the truth.) We already teleport documents (by FAX) and CADCAM (computer-aided-design/computer-aided-manufacture) would readily permit us to teleport automobile parts. Is it all living things, or only all complicated living things, or only all human brains that quantum mechanics would prevent from being thus teleported? Are we the only things in the universe that require quantum computers for their persisting identity?

For those who share Penrose's suspicion about human teleportation, this is one of the comforting
implications of his theory, but the price to be paid (in terms of revision of the Cathedral of
Science) is high. Among the likely casualties, according to a tentative and impressionistic
argument of Penrose, will be the standard neo-Darwinian theory of evolution by natural selection.
He does not see how algorithms for mathematical judgment could evolve, and "To my way of
thinking, there is still something mysterious about evolution, with its apparent 'groping' towards
some future purpose. Things at least *seem* to organize themselves somewhat better than they
'ought' to, just on the basis of blind-chance evolution and natural selection." (p.416) Creationists
are not alone in harboring deep misgivings about the standard view; there are biologists who dare
to wonder about the one leap of faith still required by the standard view: has there really been
enough time for evolution to do all the designing by blind methods? Penrose shares those doubts,
but provides no new argument to support them.

Might Penrose be right about all this? I suppose he might; he is an extraordinarily inventive and undogmatic thinker with an awesome mastery of many fields. If anyone could make such a discovery, it would have to be someone like Penrose. But whether he is right or not, his strenuous efforts to combat strong AI by unsettling the Cathedral of Science show, more clearly than any of the manifestos for the other side, that strong AI is a straightforward implication of orthodoxy. We cannot simply add a new wing to the Cathedral, enshrining an alternative theory of the mind; if strong AI is mistaken, the whole structure of science must be rebuilt from the ground up. This will inevitably lead some readers to reason as follows: If an opponent as brilliant and dedicated as Penrose discovers he has to go to such lengths to build a presentable case against strong AI, and can come up with nothing stronger than a speculative suggestion about quantum gravity, strong AI must be more secure than I had thought!

The argument Penrose unfolds has more facets than my summary can report, and it is unlikely that such an enterprise would succumb to a single, crashing oversight on the part of its creator--that the argument could be "refuted" by any simple objection. So I am reluctant to credit my observation that Penrose seems to make a fairly elementary error right at the beginning, and at any rate fails to notice or rebut what seems to me to be an obvious objection. Recall that the burden of the first part of the book is to establish that minds are not "algorithmic"--that there is something special that minds can do that cannot be done by any algorithm (i.e., computer program in the standard, Turing-machine sense). What minds can do, Penrose claims, is see or judge that certain mathematical propositions are true by "insight" rather than mechanical proof. And Penrose then goes to some length to argue that there could be no algorithm, or at any rate no practical algorithm, for insight.

But this ignores a possibility--an independently plausible possibility--that can be made obvious by
a parallel argument. Chess is a finite game (since there are rules for terminating go-nowhere
games as draws), so in principle there is an algorithm for either checkmate or a draw, one that
follows the brute force procedure of tracing out the immense but finite decision tree for all
possible games. This is surely not a practical algorithm, since the tree's branches outnumber the
atoms in the universe. Probably there is no practical algorithm for checkmate. And yet programs--algorithms--that achieve checkmate with very impressive reliability in very short periods of time
are abundant. The best of them will achieve checkmate almost always against almost any
opponent, and the "almost" is sinking fast. You could safely bet your life, for instance, that the
best of these programs would *always* beat me. But still there is no *logical* guarantee that the
program will achieve checkmate, for it is not an algorithm *for* checkmate, but only an algorithm
*for* playing legal chess--one of the many varieties of legal chess that does well in the most
demanding environments. The following argument, then, is simply fallacious:

(1) X is superbly capable of achieving checkmate.

(2) There is no (practical) algorithm guaranteed to achieve checkmate.

therefore

(3) X does not owe its power to achieve checkmate to an algorithm.

So even if mathematicians are superb recognizers of mathematical truth, and even if there is no
algorithm, practical or otherwise, for recognizing mathematical truth, it *does not follow* that the
power of mathematicians to recognize mathematical truth is not entirely explicable in terms of
their brains executing an algorithm. Not an algorithm *for* intuiting mathematical truth--we can
suppose that Penrose has proved that there could be no such thing. What would the algorithm be
for, then? Most plausibly it would be an algorithm--one of very many--for *trying to stay alive*, an
algorithm that, by an extraordinarily convoluted and indirect generation of byproducts,
"happened" to be a superb (but not foolproof) recognizer of friends, enemies, food, shelter,
harbingers of spring, good arguments--and mathematical truths!

Chess programs, like all "heuristic" algorithms, are designed to take chances, to consider less than
all the possibilities, and therein lies their vulnerability-in-principle. There are many ways of taking
chances, utilizing randomness (or just chaos or pseudo-randomness), and the process can be
vastly sped up by looking at many possibilities (and taking many chances) at once, "in parallel".
What are the limits on the robustness of vulnerable-in-principle probabilistic algorithms running
on a highly parallel architecture such as the human brain? Penrose neglects to provide any
argument to show what those limits are, and this is surprising, since this is where most of the
attention is focussed in artificial intelligence today. Note that it is *not* a question of what the in-principle limits of algorithms are; those are simply irrelevant in a biological setting. To put it
provocatively, an algorithm may "happen" to achieve something it cannot be advertised as
achieving, and it may "happen" to achieve this 999 times out of a thousand, in jig time. This
prowess would fall outside its official limits (since you cannot prove, mathematically, that it will
not run forever without an answer), but it would be prowess you could bet your life on. Mother
Nature's creatures do it every day.

I may well have missed a crucial ingredient in Penrose's argument that somehow obviates this criticism, but it is disconcerting that he does not even address the issue, and often writes as if an algorithm could have only the powers it could be proven mathematically to have in the worst case. It will be interesting to see how he would repair this omission. In the meantime I would say that whether or not the Penrose revolution in physics is coming, he has not yet shown the need for the revolution in order to explain facts of human cognitive competence.

I have left no doubt about the difficulty of this book, and I must balance that impression by noting
that it is nevertheless a pedagogical *tour de force*, with some dazzling new ways of illuminating
the central themes of science. I was struck as never before by the gleeful staircase of human
artifices--diagrams, mappings, formalisms--piled one on top of the other over the years, permitting
our species so much as to *entertain* such audacious hypotheses about the world we live in. His
discussion of phase spaces, for instance, and his development of the rationale for the second law
of thermodynamics, are particularly refreshing. His exemplary candor, particularly in the chapters
on cosmology and quantum physics, provides the uninitiated reader with a vivid experience of the
way gut intuitions and aesthetic reactions call the tune in science *until* someone figures out a
conversation-stopping proof, mathematical or experimental.

And along the way he makes important points that have been overlooked by the believers in strong AI, even if they can be incorporated into it. For instance, he closes the book with a speculation about time I believe is exactly right:

I suggest that we may actually be going badly wrong when we apply the usual physical rules fortimewhen we consider consciousness! . . . My guess is that there is something illusory here. . .and the time of our perceptions does not 'really' flow in quite the linear forward-moving way that we perceive it to flow (whatever that might mean!). The temporal ordering that we 'appear' to perceive is, I am claiming, something that we impose upon our perceptions in order to make sense of them in relation to the uniform forward time-progression of an external physical reality. (p.443-4)

This is, in my opinion, the key to removing the last, harmful vestiges of Cartesian thinking from our standard vision of how consciousness relates to the brain, but you don't need quantum magic or quantum gravity to get there. A clear statement of the point has been given by Douglas Snyder Endnote 1, and I myself have more recently been developing the case for this claim from an entirely conservative--indeed an "engineering"--base, as the best way for Mother Nature to handle the synchronization problems that arise in a brain that must cope with events that sometimes occur on a time scale faster than its own internal transmissions. Endnote 2

A philosophy professor once said to his class, "I want you to believe the things I tell you, but not
because you *believe me*; I want you, rather, to believe them because you yourself see that they
must be true." This is Penrose's ideal, and indeed it should be every teacher's ideal, but we all fall
short; the semester (or life) is too short, and at some point we fall back on "Take it from me: that
idea just doesn't work." Penrose is positively heroic in his attempts to live by this standard. The
reader is warned, *after *weathering over two hundred pages on the lambda calculus, the class of
NP-complete problems, Maxwell's equations, the Lorentz equation, special and general relativity,
and much more, that in the next chapter, on quantum mechanics, things are going to get "a bit
technical. In my descriptions I have tried not to cheat, and we shall have to work a little harder
than otherwise." (p.227) But although matters do then get still more technical, uninitiated readers
cannot "work harder"--because we simply do not know the rules. If we are to "see for ourselves"
the truths of quantum physics, we must be active and skeptical, but the world of quantum physics
is so alien that we can no longer trust our untutored judgments of what counts as a telling
objection and what is merely a misapplied maxim or analogy drawn from more familiar territory. I
suspect that nothing short of extended immersion in the actual use of the mathematics to solve
particular problems can give one a confident sense of how this game is played, and why the rules
are what they are. We get assured by Penrose that various hard-to-swallow options make sense
while others are just not on, and we have to take his word for it. His brilliant exposition up to this
point gives us ample reason to respect his *obiter dicta* once they start to flow, but, contrary to his
best intentions, his readers at this point must cease being participants and start being spectators.

This raises a perplexity about Penrose's intentions in writing this book. He repeatedly
acknowledges that his colleagues, who already understand the difficult materials he is teaching us
much better than we ever will, do not yet accept his idiosyncratic vision. But if he can't convince
*them*, pulling out all the stops, what good will it do if he convinces *us* with a relatively elementary
version? What then? Are we supposed to join in a Children's Crusade to persuade his colleagues
to get in step? This cannot be his intention.

I suspect he has a more subtle strategy in mind. When experts talk to experts, they are careful to
err on the side of *under*explaining the fundamentals. One risks insulting a fellow expert if one
spells out very basic facts. There is really no socially acceptable way for Penrose to sit his
colleagues down and lecture to them about their oversimplified and complacent attitudes about
fundamentals. So perhaps educated laypeople are only the presumptive audience for this book,
hostages to whom he can seem to be addressing his remarks, so that the experts--his real target
audience--can listen in, from the side, without risk of embarrassment. I think this is a wonderful
strategy, perhaps the only way of getting experts who are talking past each other to refresh their
mutual understanding of the fundamentals. (It is especially valuable in philosophy.) It may leave
the non-experts in the role of spectators, but at least it gives them ringside seats.

1. "On the Time of a Conscious Peripheral Sensation," *Journal of Theoretical Biology*, 1988,
**130**, 253-254.

2.In "The Autocerebroscope," at a symposium in memory of Heinz Pagels, The Reality Club, Alliance Française, New York City, February 1, 1989; "Temporal Anomalies and the Architecture of Consciousness," Cognitive Science Colloquium, Indiana University, February 28, and the Gildea Lecture at Washington University School of Medicine, May 2, 1989.