Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Path: utzoo!watmath!clyde!caip!nike!ucbcad!ucbvax!NBS-VMS.ARPA!cugini From: cugini@NBS-VMS.ARPA ("CUGINI, JOHN") Newsgroups: mod.ai Subject: Searle and Understanding Message-ID: <8607181854.AA27962@ucbvax.Berkeley.EDU> Date: Tue, 15-Jul-86 08:16:00 EDT Article-I.D.: ucbvax.8607181854.AA27962 Posted: Tue Jul 15 08:16:00 1986 Date-Received: Sat, 19-Jul-86 03:47:10 EDT Sender: daemon@ucbvax.BERKELEY.EDU Reply-To: "CUGINI, JOHN" Organization: The ARPA Internet Lines: 55 Approved: ailist@sri-ai.arpa This is in response to recent discussion about whether AI systems can/will understand things as humans do. Searle's Chinese room example suggests the extent to which the implementation of a formal system may or may not understand something. Here's another, perhaps simpler, example that's been discussed on the philosophy list. Imagine we are visited by ETS - an extra-terrestial scientist. He knows all the science we do plus a lot more - quarks, quantum mechanics, neurobiology, you-name-it. Being smart, he quickly learns our language and studies our (pitifully primitive) biology, so he knows about how we perceive as well. But, like all of his species, he's totally color-blind. Now, making the common assumption that color-knowledge cannot be conveyed verbally or symbolically, does ETS "understand" the concept of yellow? I think the example shows that there are two related meanings of "understanding". Certainly, in a formal, scientific sense, ETS knows (understands-1) as much about yellow as anyone - all the associated wavelengths, retinal reactions, brain-states, etc. He can use this concept in formal systems, manipulate it, etc. But *something* is missing - ETS doesn't know (understand-2) "what it's like to see yellow", to borrow/bend Nagel's phrase. It's this "what it's like to be a subject experiencing X" that eludes capture (I suppose) by AI systems. And I think the point of the Chinese room example is the same - the system as a whole *does* understand-1 Chinese, but doesn't understand-2 Chinese. To get a bit more poignant, what systems understand-2 pain? Would you really feel as guilty kicking a very sophisticated robot as kicking a cat? I think it's the ambiguity between these senses of understanding that underlies a lot of the debate. They correspond somewhat to Dennett's "program-receptive" and "program-resistant" properties of consciousness. As far as I can see, the lack of understanding-2 in artificial systems poses no particular barrier to their performance. Eg, no doubt we could build a machine which in fact would correctly label colors - but that is not a reason to suppose that it's *conscious* of colors, as we and some animals are. Nonetheless, *even if there are no performance implications*, there is a real something-or-other we have going on inside us that does not go on inside Chinese rooms, robots, etc., and no one knows how even to begin to address the replication of this understanding-2 (if indeed anyone wants to bother). John Cugini ------