Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!iuvax!cogsci!dave From: dave@cogsci.indiana.edu (David Chalmers) Newsgroups: comp.ai Subject: Re: Question on Chinese Room Argument Summary: On the nature of symbols. Message-ID: <17923@iuvax.cs.indiana.edu> Date: 25 Feb 89 03:02:31 GMT References: <4298@pt.cs.cmu.edu> <4296@cs.Buffalo.EDU> <1989Feb20.213329.10376@cs.rochester.edu> <855@jhunix.HCF.JHU.EDU> Sender: root@iuvax.cs.indiana.edu Reply-To: dave@duckie.cogsci.indiana.edu (David Chalmers) Organization: Concepts and Cognition, Indiana University Lines: 119 The discussion on Searle's Chinese room seems to be becoming very confused. (I could make some bad jokes about the 'understanding' of some of the participants, but I'll refrain.) I thought I'd try to clear up one of the main sources of confusion, the word "symbol." This word is being used to mean two different things: (1) A "symbol" is a formal object which corresponds to some HIGH-LEVEL, semantic concept in the real world. Typically the concept which it corresponds to is on the level of a _word_, say, as opposed to a microstructural level such as that of a neuron. versus (2) A "symbol" is any formal object which is manipulated by a computer program. What we take this symbol to correspond to may be as low-level or as high-level as we like, or we may decide that the question of what the symbol corresponds to is meaningless and unimportant. Sense (1) is the sense in which the word "symbol" is used most of the time in AI. Newell, Simon, Fodor et al are all supporters of the "Symbolic Paradigm", which essentially means that they claim that true AI could be achieved by a program which formally manipulates such high-level symbols. (I'll say that by "true AI" here I mean a program which displays intelligent behaviour, in order to forestall questions like "but is it really thinking?"). Many people these days dispute this claim. One of the main reasons is that denoting such high-level, complex and inherently semantic concepts by a rigidly syntactic formal object will never be able to capture the richness and flexibility of such concepts. In a sense, these formal symbols are brittle and empty, devoid of "meaning." This is clearly also the sense of "symbol" which people have been using when they speak of the difficulty of understanding Portuguese or physics or mathematics by using pure symbol manipulation rules. Despite the fact that with such rules one can reproduce vaguely competent behaviour, the rigidity of such rules can, I believe, always be detected by close questioning, or observation under new and unusual circumstances. The fact is that high-level concepts interact in far too rich and flexible a manner, and this richness could never be captured by a set of rules which manipulate concepts as chunks. So I say: with sense (1) of "symbol", even weak AI can never be achieved. It will be impossible to fully reproduce intelligent behaviour. Thus, with this sense of "symbol", I reject the PREMISE of Searle's argument; a formal symbol-manipulator could never even display what _looked_ like competent Chinese-speaking behaviour. Thus, of course I am with Searle here in saying that such symbol-manipulators could never have true (subjective) understanding. But for me it's not an issue, for I believe that such manipulators would never even LOOK as if they understood. If this was Searle's point, this would be fine. But Searle wants to claim more. Contrary to what Harnad implies, Searle is not only arguing against high-level symbol manipulators in the Newell/Simon/Fodor mould. He wants to say that NO computer program could ever be enough to have true (subjective) understanding, not even an incredibly complex and subtle program (such as a program that simulated a neural network the size of the brain.) To do this, Searle uses the word "symbol" in sense (2), where it can denote any formal object whatsoever that is manipulated. The symbol can correspond to something as low-level as a neuron, or it may correspond to something which on the face of it has no meaning to us whatsoever. Presumably in a neural-net-simulator, a symbol corresponds to a neuron or one of its constituent parts - not a very 'semantic' object at all! But here is Searle's trick, or to be charitable (or uncharitable?) his mistake. He uses the word "symbol" in the low-level sense (2), while appealing to our intuitions about symbol-manipulators which manipulate high-level symbols of sense (1)! He says, (paraphrasing), "such a formal manipulator could never capture the SEMANTICS of the world to which the symbols correspond." What he implies here is that the symbols correspond to objects which have meaning, but that formal manipulation can never capture that meaning. (Just as those Brazilian physics students manipulated equations without anyone knowing what they _meant_.) But AHA - here we have him. These low-level (sense 2) symbols never had much meaning anyway! They correspond to micro-structural entities (such as neurons), which taken alone are devoid of semantics. Semantics only emerges when we put enough of these neurons together to form an incredibly complex SYSTEM. In a precisely similar way, semantics (and hence understanding) will arise from our sense-2-symbol-manipulator, when it has enough of these low-level symbols interacting in the right, incredibly complex way. Despite the fact that the symbols taken alone are meaningless, put enough of them together in the right way and meaning will be an EMERGENT property of the system, just as it is with the human brain. So this is the source of Searle's mistake. He appeals to our intuitions about high-level (sense 1) symbol-manipulators, and tries to use this to draw conclusions about low-level (sense 2) symbol manipulators. And by doing this he fails to appreciate the incredible complexity and subtlety that is possible in a sense-2-manipulator, from which understanding can be an emergent property, as it is in the human brain. It is a very mysterious question indeed how real understanding, subjective experience and so on could ever emerge from a nice physical system like the human brain, which is just toddling along obeying the laws of physics. But nevertheless we know that it does, although we don't know how. Similarly, it is a mysterious question how subjective experience could arise from a massively complex system of paper and rules. But the point is, it is the SAME question, and when we answer one we'll probably answer the other. I'll resist the temptation to answer each of Searle's other points one by one. Just remember, semantics CAN arise from syntax, as long as the syntactical system is complex enough, and involves manipulating micro-structural objects which interact in rich and subtle ways. So, for you neural-netophiles out there (as well as the rest of us fellow subcognitivists), there's hope yet! (Just keep the discussion of symbols on the right level.) Dave Chalmers Center for Research on Concepts and Cognition Indiana University