Path: utzoo!censor!geac!torsqnt!news-server.csri.toronto.edu!cs.utexas.edu!know!sdd.hp.com!ucsd!ames!eos!shelby!msi.umn.edu!cs.umn.edu!thornley From: thornley@cs.umn.edu (David H. Thornley) Newsgroups: comp.ai.philosophy Subject: Re: Searle's Chinese Room Keywords: Strong AI, Turing test Message-ID: <1990Nov22.022129.18358@cs.umn.edu> Date: 22 Nov 90 02:21:29 GMT References: <16197@mentor.cc.purdue.edu> <3952@media-lab.MEDIA.MIT.EDU> <10297@jpl-devvax.JPL.NASA.GOV> <1990Nov16.171041.14144@ncsuvx.ncsu.edu> <1990Nov19.192555.29337@cs.umn.edu> <1990Nov21.181445.11552@n Organization: University of Minnesota, Minneapolis - CSCI Dept. Lines: 136 In article <1990Nov21.181445.11552@ncsuvx.ncsu.edu> fostel@eos.ncsu.edu (Gary Fostel) writes: > >I few posts back, I made the observation that the Turing test was not >a well defined test at all because it does not pin down who's judegement >is to be used in deciding if the test has been passed. No doubt Turing >was expecting it to be conducted by himself, but with him gone, I wonder >who will decide? David Thornley, at the University of Minnesota replied: > > It is reasonably well-defined. Read Turing's paper. (Please, everyone, > read and ponder both one or both of Searle's articles, and Turing's paper, > before jumping in. Searle gives me the impression that he is arguing > against something of a straw Turing test.) Sorry, that's not the impression I got. This may well have been my mistake; I see lots of misconceptions. > >I have read Turing original description, and also some of his other musings >on the use of similar tests to see if men and women could tell each other >apart by linguistic means. I spent quite some time discussing it with >AI people and technical philospohers and my conclusion about the weakness >of the test is a strongly held one. Whether or not a machine can pass the >Turing Test ought not to be a function of the judgement of the person who >is trying to apply the test. > >To illustrate this point, I used a trivial analogy of a test for chess >playing similar to the Turing test for "intelligence". The point is that >experts in chess, computing, or both, will be far better able to recognize >a computer program playing chess, than will the average person. If we have >a Chess-Test, constructed as the Turing test, then do we accept the judgement >of the experts who can recognize oddities of the computer programs or do we >accept the judegement of an average person? To be more hip, perhaps I should >change this to Chinese Chess and put the chess playing agent in a room. Then >we can all debate Fostel's Chinese Chess Room. > >Thornley went on to say: > > The Turing test is not to convince the observer of intelligence, but to > be *indistinguishable* from an intelligent adult human. If you put a > standard chess program on line, I can distinguish it immediately by asking > about the situation in the Middle East. > >This seems to miss the point I was making. Perhaps Thornley should "read >and ponder" my words before jumping in. The analogy to the chess playing >test is simply a way to amplify the ambiguity present in the Turing test >by using a structurally identical test of properties we understand better. >The same problems exist in the Turing test. Computer programs already exist >that have been confused with intelligent humans, e.g. Weizebaum's Doctor >and the "Paranoid" program (from Stanford?) Observers with different >background (i.e. people on this newsgroup) would not be so easily fooled, >but whose judegement it to be used? > I have not heard that Doctor or Paranoid have passed the Turing test as Turing specified it. Further, let's drop the "Paranoid" test. I can write a version of Catatonic that is guaranteed indistinguishable from a genuine catatonic. >How can the issue of individual judgement be eliminated from the Turing >test? Well, that's the point, it can not. Perhaps one could formulate a >democratic Turing test, and use the consensus of observers to decide if >the agent "passed". Or the Genius Turing test, and have the observer with >the highest IQ make the judgement. Or go for consensus: every observer >must agree. The latter is the only one that make sense. I do not believe >any agent would ever pass that test, if only becuase one or more observers >did not like the personality/politics or lingustic style of the agent. >Remember, we are looking to distinguish a computer from an intelligent human. > >How intelligent? Suppose the agent a very stupid and ill-informed human. >They may have no opinion on the current going-on in the mideast, may have a >3rd grade vocabulary and after a while get mad and refuse to co-operate. > Reread the paper. Look at the example dialogs Turing gives. These aren't the dialogs from an ignorant, inarticulate, human. While I don't remember Turing specifying an intelligent adult, I don't remember him specifying typing ability either. Use a little common sense. >[More on this point. How about a Turing Test for the stupid and > inarticulate?] > >It's worth recalling that Turing originally made up the test in the context >of being able to distinguish a person of sex A from a person of sex B >pretending to be a person of sex A. I wonder if Turing himself took it >as seriously as people think. If I imagine him looking down from heaven, >on the current debate, I think he would be chortling in that annoying >high-pitched nasal voice that earned him so many friends. He probably >had the forsight to know that programs like Doctor were feasible and he >was anxious to weaken the then-common dualist belief that humans were the >uniquely chosen vessel of intelligence. Of course, he was an atheist, >so if he IS looking down from heaven, the jokes on him! > I'm not completely sure about the Imitation Game myself. (This is the version where there is a man and a woman instead of a computer and a human. The man's job is to convince the interrogator that he is a woman; the woman's job is to convince the interrogator that she is a woman. The analogy is exact if you require the interrogator to be a woman.) As far as the definition goes, I will admit that Turing didn't write up the experimental technique like a description in a psychology paper, but the necessary elements are there. If I were to suggest a test for the hypothesis that concrete words are better remembered than abstract, I might say something like "Take some subjects. Give them a list of concrete and abstract nouns combined. Give them recognition and recall tests." I would expect the experimenter to know that subjects should speak the language the words are in. Unfortunately, far too much of the arguments surrounding the Turing test have tried to distort the test in various ways. Did Turing have to explicitly specify that an intelligent and cooperative adult with reasonable typing ability was to be used as the human in the comparison? Do I have to specify that the words should be taken from the subject's native language? So who should judge? I wouldn't trust a psychologist to know how a computer acts, a police officer might consider the psychologist and me to lack practical experience with people's reactions, the list goes on. How about taking various groups of people, such as psychologists, computer scientists, cognitive scientists, police officers, social workers, politicians, any sort of group with some expertise either with people or with computers, and see if any group has an above-chance ability to tell the human from the computer? I'd ask for confidence measures, myself. Personally, I don't think Turing thought it was very important who should judge. I think he considered that any computer that could fool a large number of informed and intelligent observers should be thought of as passing the test. If you want something more specific, he thought that, by 2000, computers with giga-something (byte? digit?) memory would do so well that an average interrogator would identify the human only 70% of the time after five minutes' conversation, and he thought that, by then, people would agree that machines could think. (If you want a specific answer to "who should interrogate," I think this satisfies your requirements.) DHT