Path: utzoo!censor!geac!torsqnt!news-server.csri.toronto.edu!cs.utexas.edu!usc!wuarchive!mit-eddie!bbn.com!BKort
From: BKort@bbn.com (Barry Kort)
Newsgroups: comp.ai.philosophy
Subject: Re: intelligence definition, reasons for
Message-ID: <61528@bbn.BBN.COM>
Date: 12 Dec 90 16:02:34 GMT
References: <14759@sdcc6.ucsd.edu>
Sender: news@bbn.com
Organization: BBN Labs
Lines: 35

In article <14759@sdcc6.ucsd.edu> jude@sdbio2.ucsd.edu (Jude Poole) writes:

> We would certainly want to know if some purportedly self aware computer
> performing a critical task 'asked' to be relieved of the task because it
> didn't want to do what it was doing so because it was really sentient or
> merely had been programmed by some malcontent to do so.  I personnally
> don't believe you can create awareness in a computer at all.  My reasons
> are largely philosophical and the 'Turing test' does nothing at all to
> address them.  To me a machine that passes the 'Turing test' is a
> tremendous accomplishment and a vindication of the pie-in-the-sky AI
> types, but ultimately in ethical terms I'll consider such a thing to be
> merely an interesting machine.

Jude, suppose you had a computer that developed its own goals by comparing 
the current state of affairs to a possible future state of affairs, and 
elected courses of action intended to evolve the current state toward more 
desirable goal states.  Such a computer would necessarily be operating in 
accordance with a Value System, which I will elucidate in  a moment.  But 
in terms of the mechanics, such a computer would rely extensively on 
model-based reasoning to anticipate the likely consequences of alternative 
courses of action in its search for for viable, practical, and effective 
strategies.  We already have good chess-playing computers who illustrate the 
mechanics of this process albeit in a limited domain.

Now let's turn to the philosophically more interesting question of imbuing 
computers with Value Systems.  A science-minded computer would have a high 
regard for information, knowledge, insight, understanding, and world-models.  V'ger in Star Trek was such a computer.  So knowledge might be a highly prized value, and knowledge-seeking behavior would result.  We know that some bio-computers value power (ownership and control) so we might expect to find some computers to imitate those values.   It is an interesting excercise in philosophy to define a Value System for computers of the future, and I will stop here and let others ponder this issue before comm







enting further.

Let me just end by pointing out that this bio-computer has on occasion refused to complete an assigned task because of an intuition that it would lead to undesirable downstream consequences.  I would hope that future computers, with comparable or superior powers of model-based reasoning would be equally recalcitrant.

Barry Kort
Visiting Scientist
BBN Labs
Cambridge, MA