Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!wuarchive!zaphod.mps.ohio-state.edu!usc!venera.isi.edu!smoliar
From: smoliar@vaxa.isi.edu (Stephen Smoliar)
Newsgroups: comp.ai
Subject: Re: Some Sequence Prediction Work (Long) Re: pattern recognition
Summary: some thoughts on the role of memory in sequence prediction
Keywords: patterns
Message-ID: <10804@venera.isi.edu>
Date: 4 Dec 89 17:13:27 GMT
References: <2850@ucrmath.UCR.EDU> <11883@phoenix.Princeton.EDU>
Sender: news@venera.isi.edu
Reply-To: smoliar@vaxa.isi.edu.UUCP (Stephen Smoliar)
Organization: USC-Information Sciences Institute
Lines: 66


The points introduced in McCarthy's critique of sequence extrapolation are well
taken;  but it is important to recognize that he is basically criticizing the
"standard" formulation of the problem.  However, this comes close to beating
a dead horse (or, at least, a horse which should have died some time ago),
since intelligent agents seldom (if ever) attempt to analyze any given set
of input stimuli in a vacuum.  I think what is important about the problems
raised by Wittgenstein and Kripke is that there are no properties strictly
inherent in any sequence which serve as grounds to predict its continuation.

What, then, is extrapolation REALLY about?  I would argue that it is really
about memory.  When an intelligent agent encounters the sequence 2, 4, 6, 8, ...
he recognizes it as a familiar sequence.  If he has ANY experience in playing
around with integers (and do any of us lack such experience entirely?), the
sequence will be familiar enough for him to recall that 10 is the next element.
Of course, he also knows enough about the real world to know that he only
EXPECTS 10 to be the next element.  If the next element turns out to be 57,
he will probably be quite surprised;  however, he is unlikely to react in panic
that the world around him is falling apart.

The point is that the best you can do by way of prediction is to model the
expectations of some particular agent.  You can, if you wish, assume that
the agent begins with no memories at all;  but you had better then prepare
yourself for a rather substantial training period before you get any
interesting behavior.  I am more curious about what it would mean to
build an agent who already has some memories.  How would such memories
be modeled?  How might I build and compare different agents who have different
bases of experience?

Many builders of expert systems will probably argue that this is a poor way to
view the world.  "The experience is coded in the rules," they would argue.
"All you have to do is find an expert and do knowledge engineering on him.
Then all of his memories and experiences will be implemented in your rule
base."  Unfortunately, I would view such a response as evasive.  When all
is said and done, such systems are not particularly good when it comes to
augmenting their knowledge on the basis of further experience.  There are,
of course, some promising advances in machine learning;  but I would say
that we are still a far cry from a system which, once endowed with some
set of expert rules for sequence extrapolation, would then "recognize"
3, 1, 4, 1, 5, 9, ... as being "familiar" and "expect" 2 on the basis
of that familiarity.  (If this example is too "universal," consider an
agent who is given six digits which happen to be the first six digits
of his home telephone number.)

I happen to share Eliot's interest in music.  I believe that listening to music
is an activity which is guided strongly by the expectations of the listener.
(Quite often, your average listener will complain about listening to Webern
because what he is hearing does not conform to expectations he has formed on
the basis of listening to, say, Tchaikovsky.)  Those expectations, then, are
founded on past listening experiences;  so in order to figure our how they are
formed, we must first figure out how one accumulates and retrieves memories of
those experiences.  (This point of view is not that different from that
expressed by Schank in DYNAMIC MEMORY as applied to our comprehension of
language.)

=========================================================================

USPS:	Stephen Smoliar
	USC Information Sciences Institute
	4676 Admiralty Way  Suite 1001
	Marina del Rey, California  90292-6695

Internet:  smoliar@vaxa.isi.edu

"For every human problem, there is a neat, plain solution--and it is always
wrong."--H. L. Mencken


Brought to you by Super Global Mega Corp .com