Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!uflorida!haven!aplcen!jhunix!ins_atge
From: ins_atge@jhunix.HCF.JHU.EDU (Thomas G Edwards)
Newsgroups: comp.ai.neural-nets
Subject: Re: NN Question
Summary: NN's as math
Message-ID: <1163@jhunix.HCF.JHU.EDU>
Date: 16 Mar 89 21:44:38 GMT
References: <32125@gt-cmmsr.GATECH.EDU> <8903071701.AA12290@shire.cs.psu.edu> <11114@pasteur.Berkeley.EDU> <10192@nsc.nsc.com>
Reply-To: ins_atge@jhunix.UUCP (Thomas G Edwards)
Organization: The Johns Hopkins University - HCF
Lines: 39

In article <10192@nsc.nsc.com> andrew@nsc.nsc.com (andrew) writes:
[concerning clocked NN's]

There is a big concern over synchronicity of NN's.  Two points come to mind,
1) Back-prop in particular is an approximation of gradient-descent of
   the error surface, and there are a few problems caused by finitely
   small quanta of learning steps...but that's what you get for not
   spending the time to search the entire error surface!
   But it would be nice if a method can be determined which allows for
   infinitely-small learning steps at a reasonable speed.
   Pineda claims his recurrent learning algorithm is "presented
   in a formalism appropriate for implementation as a physical
   nonlinear dynamical system," and thus he is able to avoid
   "certains kinds of oscillations which occur in discrete time
   models usually associated with backpropogation."  

2) To a limited extent, using "delay neurons," a syncrhonous neural
   network can approach a non-synchronous one.

>While I'm here, I'll mention something else from biology, which filled me
>with great dismay(!) - this month's Scientific American's feature on the
>brain's star-like "astrocyte" cells. Their role becomes important in direct
>proportion to the amount of time they are investigated; akin to glial cells,
>I believe.

Ah, the important thing to remeber is that NN's are based upon mathematical
solutions to the problem of getting the proper output from a network
for a certain input by changing the network weights...they might at some
level of abstraction resemble real neural networks, but lack
neuropharmacology (which is _very_ important to human cognition!),
and a whole host of other qualities.  (The brain also has many different
styles of neurons!).  
   This is _not_ to say that human brain study is irrelevent to NN's,
but that NN's are going to be a simpler structure than the brain
because they exist (currently...this may change) in the realm of
information instead of being physical things which need support,
oxygen, nutrients, immune systems, etc.

-Thomas Edwards