Commentary on Pylyshyn
Daniel C. Dennett
Center for Cognitive Studies
Medford MA 02155
Does your brain use the images in it, and if so, how?
ABSTRACT: The presence of spatial patterns of activity in the brain is suggestive of image-exploiting processes in vision and mental imagery, but not conclusive. Only behavioral evidence can confirm or disconfirm hypotheses about whether, and how, the brain uses images in its information-processing, and the arguments based on such evidence are still inconclusive.
Nobody denies that when we engage in mental imagery we seem to be making pictures in our heads–in some sense. The question is: are we really? That is, do the processes occurring in our brains have any of the properties of pictures? More pointedly, do those processes exploit any of the properties of pictures? When you make a long-distance telephone call, there is a zigzaggy pattern of activity running through various media from you to your listener across the country, but if the curves and loops and angles happened to spell out “Happy Birthday” (as seen from space), this would be an image on the surface of the planet that was not exploited in any way by your information-transmission, even if it was a birthday greeting.
Consider the mindless doodling that involves filling in all the closed loops on a printed page with your pen. The word “doodling” would get ink on every letter except the “lin” group, wouldn’t it? That process depends only on the image properties of the text, and not at all on the meaning, or even on the identity of the symbols. You can perform it just as easily on printed text in any language. Now consider spell-checking. You can’t spell-check a bit-map picture of a page of text. You have to run it through OCR (optical character recognition) first, changing the categories from shapes of black-on-white to strings of alphabet characters. Is the resulting data structure an image or not? Since there is no canonical and agreed-upon list of image-exploiting processes, this is an ill-posed question. In some ways it is (like) an image and in some ways it is not. The processes that can extract depth in a random dot stereogram are, like the doodling process, strongly imagistic–in one sense: they are totally dependent on the topographical properties of the pattern of stimulation, and not at all on the content thereby represented (there being none until after the shape in depth has been extracted). The processes that can “rotate images” a la Shepard and Metzler are, in contrast, strongly dependent on previously extracted content (try rotating half a random-dot stereogram), so they are a bit more like spell-checking, a bit less a matter of “brute” image-processing.
What sometimes looks like deliberate shifting of the goal-posts in the long-running debate over imagery is better seen, I think, as the gradual clarification of the ill-defined question above. But confusion and talking-past-each-other persists. As Pylyshyn stresses, the evidence from neuroimaging studies is, so far, almost irrelevant to the points of contention. The presence of readable images of activity in the brain is suggestive of image-exploiting processes, probably a practical necessity for such processes to occur, but not conclusive. As Kosslyn (1994, p80) notes, in the long passage quoted by Pylyshyn, the issue is about a functional space, not necessarily a physical space. It’s like computer graphics. As long as the data structures consist of properly addressed registers over which the operations are defined, the activity can be arbitrarily scattered around in space in the computer’s memory without hindering the image-exploitation that is going on.
I explain this to my students with a little thought experiment: Dismantle a mosaic, tile by tile, numbering each space consecutively, line by line, and mailing the tiles individually to friends all over the world, writing the address to which each tile is mailed after the number on a long list (a list, not a map). The mosaic on the floor is gone, the physical image destroyed. Then ask yourself questions such as: “Are there any strings of four black tiles in a horizontal row surrounded by white?” This can be answered, laboriously, by asking all your pen pals to send a “Yes” message if they have been sent a black tile, and then analyzing the list to see if any four consecutive “Yes” answers show up, and then calculating which pan pals (the “neighbors” of those four, wherever in the world they are) to ask if their tiles are white. The physical image on the floor has been destroyed, but the information in it is all available for image-exploiting processes to work on. You might happen to mail neighboring tiles in the mosaic to friends who lived near each other, but the system’s operation does not at all depend on this coincidence, however convenient it might be in practical terms. The brain, needing to work fast with rather slow connective fibers, probably preserves as much geographical correspondence as possible–the retinotopic maps–for exploitation in such inquiry-processes.
Probably the brain’s preservation of topological relationships is no mere byproduct of thrift-in-wiring, but also an enabling condition for image-exploiting processes of information-extraction. Now, can we prove it? That is what Pylyshyn has been asking all these years, and as he says, the answers so far largely fail to come to grips with the logical requirements for a positive answer. In particular, one cannot establish that mental imagery involves image-exploiting processes by showing that it utilizes the brain’s vision systems, because it has yet to be established when and how vision utilizes image-processing! Vision isn’t television. The product of vision is not a picture on the screen in the Cartesian Theater (Dennett, 1991). The fleeting retinal images punctuated by saccades are the first images, and they are not the last, as Julesz (1971) demonstrated by showing perception of depth in random dot stereograms that requires image-processing after the optic chiasma. But which subsequent cortical processes also exploit any of the informational properties of images? The eventual “products” of vision are such things as guided hand and finger motions, involuntary ducking, exclamations of surprise, triggering of ancient memories, sexual arousal, . . . . and none of these is imagistic in any sense, so assuming that the events in their proximal causal ancestry are imagistic is rather like assuming that power from a hydroelectric plant is apt to be wetter and less radioactive than power from a nuclear plant. The raw retinal data are cooked in many ways betwixt eyeball and verbal report (for instance). How cooked are the processes involved in (deliberate or voluntary) mental imagery? We don’t know yet, though investigations are gradually peeling away the alternatives.
As Pylyshyn says, behavioral evidence–patterns of ease and difficulty, timing and vulnerability to disruption, and the like--is the only evidence that can show that, and how, the brain uses images in its activities. To organize the evidence, we use the heterophenomenological method (Dennett, 1982, 1991) to provide a neutral catalog of how it seems to subjects under many varied conditions, and then our task is to devise and confirm theories that predict and explain all that seeming.
Dennett, Daniel C., 1982, "How to Study Consciousness Empirically: or Nothing Comes to Mind," Synthese, 53, 159-80.
Dennett, Daniel C., 1991, Consciousness Explained, Boston: Little, Brown, and London:Allen Lane.
Julesz, B., 1971, Foundations of Cyclopean Perception, Chicago: University of Chicago Press.
[abstract 53 words]
[main text 1137 words]
[references 41 words]
[grand total 1231 words]