Path: utzoo!utgpu!jarvis.csri.toronto.edu!cs.utexas.edu!asuvax!ncar!boulder!ccncsu!longs.LANCE.ColoState.Edu!ld231782
From: ld231782@longs.LANCE.ColoState.Edu (Lawrence Detweiler)
Newsgroups: comp.ai.neural-nets
Subject: the baby bootstrap (determining input...)
Message-ID: <5061@ccncsu.ColoState.EDU>
Date: 4 Mar 90 04:48:59 GMT
References: <720@berlioz.nsc.com> <6603@hydra.gatech.EDU>
Sender: news@ccncsu.ColoState.EDU
Reply-To: ld231782@longs.LANCE.ColoState.Edu (Lawrence Detweiler)
Organization: Colorado State U. Center for Computer Assisted Engineering
Lines: 69

>Note that babies bootstrap, so that "knowing what to look for" becomes
>increasingly sophisticated.

This deserves a thread of its own.  This is what is so remarkable about
the human neural net, in my opinion: it doesn't have something feeding
it the "right" answers to its neurons.  The whole thing is
self-contained.  The accomplishments of conventional computational
neural nets are not so grandiose considering that they require such
intensive and explicit training.  This is not the tired point that
"computational neural nets pale in comparison to the brain" and
therefore they can never hope to rival or surpass it (begging the
question).  Rather, it is suggesting that there may be something
fundamentally different about current NNs and the one in the brain.

Sure, an artificial NN can "learn" to speak by an EXTERNAL CONTROL that
modifies its weights.  But the human can do it with an INTERNAL one. 
Can somebody invent a black box that can learn to speak (and make
sense!) solely from interaction with the outside environment?  As long
as there is anything outside the box that BYPASSES THE INPUT when it
modifies weights you don't have a neural network in the most realistic
sense.  And any network that does is about the same in concept as a
little man inside everyone's head.  There is something crucial about the
roles of interaction and association in learning that computational nets
seem to miss.

A specious argument is that babies DO have "models", namely their
parents.  But in the beginning it is all just so much input.  Babies
must "learn to learn."  How are they to know that when mommy says "say
mommy" they are to say mommy?  Is there some set of "mommy" neurons that
are being changed based on how well they say "mommy"?  The answer is
yes, but whether this is true from the BEGINNING...?

Likewise one could argue that in current computational nets there is
only input (the stimulus and correct response, so to speak).  But the
relationship between stimulus and response is firmly established by the
EXTERNALLY DERIVED learning rule.  In any real brain this is precisely
what must be learned!  It must be INTERNALLY derived.  How does one
neuron learn what influences it, what it is modeling?

I think McClelland and Rumelhart make an allusion to this (namely the
difference between computational NNs and the human one exposed by the
"baby bootstrap") in the beginning of the first volume of PDP (of the
assumption that neurons are molded by some "correct" guide, "we think
this is nearly correct...").

One might argue that the role of heredity is to create a minimal
structure that experience can build on (the kind of heredity that makes
a baby respond differently to a smile vs. frown, track an object with
eyes, etc.).  Permit me some indulgence in the radical: suppose that
intelligence can exist INDEPENDENT of heredity!  That is, heredity is
only a way of optimizing learning (by starting from more than scratch),
but it is not necessary for learning to take place.  The development of
the child suggests this idea, which is in direct opposition to every
"supervised" learning scheme in use.  (I suggest the terms "autonomous"
or "automatic" vs. "manipulated" instead of supervised vs. unsupervised.)

The ultimate neural network would probably allow direct modification of
its weights (like in artificial models) and also "automatic" mode (like
in the brain).  You'd get the best of both worlds.  This is a convincing
argument (among others) that if an artificial brain is ever devised with
the capabilities of the human one, then there also exists one of the
former that is superior to any of the latter.

Now, don't take any of this as a criticism of neural networks in their
present form.  These are only observations.  Even though results of
learning from manipulative techniques such as back-propagation have been
less than spectacular (requiring lots of training), manipulative
techniques will probably always have a role in cases of identifiable
("right") output.