Path: utzoo!censor!geac!torsqnt!lethe!yunexus!ists!helios.physics.utoronto.ca!news-server.csri.toronto.edu!cs.utexas.edu!samsung!zaphod.mps.ohio-state.edu!caen!uflorida!gatech!ncsuvx!news From: fostel@eos.ncsu.edu (Gary Fostel) Newsgroups: comp.ai.philosophy Subject: Re: Searle's Chinese Room Keywords: Strong AI, Turing test Message-ID: <1990Nov21.181445.11552@ncsuvx.ncsu.edu> Date: 21 Nov 90 18:14:45 GMT References: <16197@mentor.cc.purdue.edu> <3952@media-lab.MEDIA.MIT.EDU> <10297@jpl-devvax.JPL.NASA.GOV> <1990Nov16.171041.14144@ncsuvx.ncsu.edu> <1990Nov19.192555.29337@cs.umn.edu> Sender: news@ncsuvx.ncsu.edu (USENET News System) Reply-To: fostel@eos.ncsu.edu (Gary Fostel) Organization: North Carolina State University Lines: 89 I few posts back, I made the observation that the Turing test was not a well defined test at all because it does not pin down who's judegement is to be used in deciding if the test has been passed. No doubt Turing was expecting it to be conducted by himself, but with him gone, I wonder who will decide? David Thornley, at the University of Minnesota replied: It is reasonably well-defined. Read Turing's paper. (Please, everyone, read and ponder both one or both of Searle's articles, and Turing's paper, before jumping in. Searle gives me the impression that he is arguing against something of a straw Turing test.) I have read Turing original description, and also some of his other musings on the use of similar tests to see if men and women could tell each other apart by linguistic means. I spent quite some time discussing it with AI people and technical philospohers and my conclusion about the weakness of the test is a strongly held one. Whether or not a machine can pass the Turing Test ought not to be a function of the judgement of the person who is trying to apply the test. To illustrate this point, I used a trivial analogy of a test for chess playing similar to the Turing test for "intelligence". The point is that experts in chess, computing, or both, will be far better able to recognize a computer program playing chess, than will the average person. If we have a Chess-Test, constructed as the Turing test, then do we accept the judgement of the experts who can recognize oddities of the computer programs or do we accept the judegement of an average person? To be more hip, perhaps I should change this to Chinese Chess and put the chess playing agent in a room. Then we can all debate Fostel's Chinese Chess Room. Thornley went on to say: The Turing test is not to convince the observer of intelligence, but to be *indistinguishable* from an intelligent adult human. If you put a standard chess program on line, I can distinguish it immediately by asking about the situation in the Middle East. This seems to miss the point I was making. Perhaps Thornley should "read and ponder" my words before jumping in. The analogy to the chess playing test is simply a way to amplify the ambiguity present in the Turing test by using a structurally identical test of properties we understand better. The same problems exist in the Turing test. Computer programs already exist that have been confused with intelligent humans, e.g. Weizebaum's Doctor and the "Paranoid" program (from Stanford?) Observers with different background (i.e. people on this newsgroup) would not be so easily fooled, but whose judegement it to be used? How can the issue of individual judgement be eliminated from the Turing test? Well, that's the point, it can not. Perhaps one could formulate a democratic Turing test, and use the consensus of observers to decide if the agent "passed". Or the Genius Turing test, and have the observer with the highest IQ make the judgement. Or go for consensus: every observer must agree. The latter is the only one that make sense. I do not believe any agent would ever pass that test, if only becuase one or more observers did not like the personality/politics or lingustic style of the agent. Remember, we are looking to distinguish a computer from an intelligent human. How intelligent? Suppose the agent a very stupid and ill-informed human. They may have no opinion on the current going-on in the mideast, may have a 3rd grade vocabulary and after a while get mad and refuse to co-operate. Are we going to judge them to NOT be a human? Not intelligent? So far as I know, we have not yet developed the capability to measure human intelligence very well; how are we going to specifify a cut-off on the Turing test for "how intelligent" the agent must be? This is another judegement call for which we are forced to rely upon the consensus of the observers. If the agent on the other end of the line is not very intelligent, do we conclude that they are not human? I hope not. But this suggests that an easy way to subvert the Turing test is to program a computer to be surley, uncommunicative, stupid and ill informed. Would it pass? What would any of this have to say about any of the profound questions of intelligence, consciousness or minds and machines? Not a heck of a lot. It's worth recalling that Turing originally made up the test in the context of being able to distinguish a person of sex A from a person of sex B pretending to be a person of sex A. I wonder if Turing himself took it as seriously as people think. If I imagine him looking down from heaven, on the current debate, I think he would be chortling in that annoying high-pitched nasal voice that earned him so many friends. He probably had the forsight to know that programs like Doctor were feasible and he was anxious to weaken the then-common dualist belief that humans were the uniquely chosen vessel of intelligence. Of course, he was an atheist, so if he IS looking down from heaven, the jokes on him! ----GaryFostel---- Department of Computer Science North Carolina State University