Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!husc6!rutgers!labrea!aurora!ames!sdcsvax!ucbvax!CORWIN.CCS.NORTHEASTERN.EDU!mckee
From: mckee@CORWIN.CCS.NORTHEASTERN.EDU
Newsgroups: comp.ai.digest
Subject: Should AI be scientific?  If yes, how?
Message-ID: <8708240436.AA19024@ucbvax.Berkeley.EDU>
Date: Fri, 21-Aug-87 13:30:39 EDT
Article-I.D.: ucbvax.8708240436.AA19024
Posted: Fri Aug 21 13:30:39 1987
Date-Received: Tue, 25-Aug-87 00:49:10 EDT
Sender: daemon@ucbvax.BERKELEY.EDU
Distribution: world
Organization: The ARPA Internet
Lines: 127
Approved: ailist@stripe.sri.com


	One reason is simple intellectual honesty.  If AI researchers
call themselves Computer Scientists (as many of them do), they're implicitly
also claiming to be scientists.  And to be perfectly blunt, any scientist
who doesn't use the scientific method is a charlatan.  I'd prefer AI to be
serious science, but if you don't want to do science, I won't argue.
Misrepresentation is a different matter:  if it's not science, don't call
it science.
	Another, more technical reason involves the perennial question
"what is reality?", and how one verifies any answer that might be submitted.
The question is important to AI not only in its "what is intelligence, really?"
aspect, but also because any AI system that interacts with the real world
ought to have an accurate understanding of its environment.  Scientific facts
are (almost by definition) the most accurate description of the universe
that we have, and scientific theories the best summaries.  And the reason
this is so is because the scientific method is the best way we've yet
discovered for making sure that facts and explanations are accurate.
	Besides science, the other significant field with aspirations toward
understanding reality is philosophy, which has even evolved a specialized
subfield, ontology, devoted to the question.  Now I haven't studied ontology,
not because the question is unimportant, but because I think philosophical
methodology is fatally flawed, and incapable of convincing me of the substance
of any conclusions that it might obtain.  I'm not interested in a discussion
of how philosophy has or has not lost its way since Kant wrote his "Prolegomena
to Any Future Metaphysics Which Will Be Able to Come Forth as Science", but
I think philosophers' methodology has kept them from being as productive of
useful understanding as they could have been.
	The critical question in choice of methodology concerns verifiablity.
I'd hate to see AI researchers cast adrift in a sea of notions by thinking
that a solid intellectual structure can be built on "Philosophical Foundations",
so I'm going to attempt to concisely describe a schema of the different ways
a theory can be confirmed.  I'm afraid I'll have to leave out a lot of details
and examples, but I hope you'll be able to fill in the rest of the picture
yourself.  In this schema, philosophy turns out to use the weakest form of
confirmation, AI as it's currently practiced uses somewhat stronger methods,
and the natural sciences end up as strongest.

	To see how this happens, think of the subject matter of a field
of study as a set of statements (observations, facts) connected by a network of
reasons.  The reasons can be arbitrarily long (or short) chains of inferences.
What a researcher needs to do to "understand" the field is find a set of
axioms and inference rules that will show the explanatory relation between
any pair of observations.  However, the problem is underdetermined -- there's
more than one consistent set of explanations for any set of facts.  At the
very least, one can always say "Because!", and define a special rule for
each ill-behaved pair of facts.  Doing this everywhere gives your theory
a very simple structure, and Occam's razor decrees that simplicity is important.
	If there are always multiple theories that can explain all the observed
data, then one must turn to some confirmation methodology to distinguish
between them, and using anything but the most powerful techniques is a waste
of time and resources.  They are all based on prediction -- applying
explanations to facts until one has covered all the facts, then generating
new "potential facts" from incompletely bound explanations.  For philosophers,
all that can be done is to compare predictions, since the operations of
the human mind are not externally visible.  Worse, the facts of experience
itself are inaccessible to more than one theorist, so that the data
can't be verified, only statements about it.  And since Godel proved his
famous incompleteness theorem, we've known that no realistic model of the
world can be derived from a finite set of axioms, so there's no way of telling
if any discrepancy in predictions might be cured by the addition of "just
one more" axiom.  [Beyond this my metamathematics doesn't go.  It would be
interesting to know if there's any convergence at higher degrees of
metafication.  I don't think so, though.]
	In AI, one can trace the operation of a theory that's been instantiated
as a program, as long as there's sharing of source code and the hardware is
the same.  This gives you operational confirmation as well as implicational
confirmation, since you can watch the computer's "mind" at work, pausing
to examine the data, or single-step the inference engine.  The points of
divergence between multiple theories of the same phenomenon can thus be
precisely determined.  But theories summarize data, and where does the
data come from?  In academia, it's probably been typed in by a grad student;
in industry, I guess this is one of the jobs of the knowledge engineer. 
In either case there's little or no standard way to tell if the data that
are used represent a reliable sample from the population of possible data
that could have been used.  In other sciences the curriculum usually includes
at least one course in statistics to give researchers a feel for sampling
theory, among other topics.  Statistical ignorance means that when an AI
program makes an unexpected statement, you have only blind intuition and
"common sense" to help decide whether the statement is an artifact of sampling
error or a substantial claim.
	In the natural sciences, in addition to implicational and operational
confirmation, you'll find external confirmation.  Each relation in the theory
is tested by an experiment on the phenomenon itself, often in many ways in
many experiments.  It's not easy to think of statements about the content
of AI (as opposed to its practice or techniques) that *could* be validated
this way, much less hypotheses that actually *have* been experimentally
validated. Hopefully, it's my ignorance of the field that leads me to
say this.  The best I can think of at the moment is "all intelligent systems
that interact with the physical world maintain multiple representations
for much of their knowledge."

	To verify a hypothesis like this, one of the strategies one can
use is to build synthetic intelligent systems and then look at their
structure and performance, remembering that the engineering used during
construction is not the scientific goal.  And then, to understand the
structure one would use analytic techniques, and to understand the performance
one would use behaviorist techniques. (Behaviorist anti-theory can safely
be ignored, but don't forget that their methodology allowed them to discover
learning sets when their animals became skilled at finding solutions to
new *kinds* of problems.)
	Another strategy is to look at the structure and behavior of the
intelligent systems one finds in nature.  One would use the same methods
to validate the behavioral descriptions as in the synthetic case, but
to study natural systems' structure one must use indirect, non-invasive means
or non-human subjects, since ethical considerations forbid destructive
testing of humans except in very special circumstances.  However the problem
here is not lack of data but lack of understanding.  If I believed that
more data was needed, I'd be back in the lab recording from multiple
microelectrodes, or standing in line for time on a magnetic resonance
imager (which can already give you sub-millimeter resolution in a 3-dimensional
brain image -- why wait for magnetoencephalography which won't tell you
what you want to know anyway?), instead of building and running abstract
models of neural tissue.


			Oops, four times as many words as I had hoped for.
			Oh well, thanks for your attention.
	- George McKee
	  College of Computer Science [sic]
	  Northeastern University, Boston 02115

CSnet: mckee@Corwin.CCS.Northeastern.EDU
Phone: (617) 437-5204

Quote of the day: "It's not what you don't know that hurts you,
			it's the things you know that ain't so."
					- Mark Twain