Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!iuvax!rutgers!tut.cis.ohio-state.edu!ucbvax!hplabs!nsc!*andrew
From: *andrew@nsc.nsc.com (*andrew)
Newsgroups: comp.ai.neural-nets
Subject: Re: NN Question
Summary: small steps and astrocytes
Message-ID: <10196@nsc.nsc.com>
Date: 17 Mar 89 07:20:42 GMT
References: <32125@gt-cmmsr.GATECH.EDU> <8903071701.AA12290@shire.cs.psu.edu> <1163@jhunix.HCF.JHU.EDU>
Organization: National Semiconductor, Santa Clara
Lines: 29

In article <1163@jhunix.HCF.JHU.EDU>, ins_atge@jhunix.HCF.JHU.EDU (Thomas G Edwards) writes:
> 1) Back-prop in particular is an approximation of gradient-descent of
>    the error surface, and there are a few problems caused by finitely
>    small quanta of learning steps...but that's what you get for not
>    spending the time to search the entire error surface!

I believe that there exists no formal proof of global convergence for 
conventional backprop when the quanta are not "infinitely small". This might
be seen as a drawback!
> 
> 
> Ah, the important thing to remeber is that NN's are based upon mathematical
> solutions to the problem of getting the proper output from a network...
> ... NN's are going to be a simpler structure than the brain
> because they exist (currently...this may change) in the realm of
> information instead of being physical things which need support,
> oxygen, nutrients, immune systems, etc.
> 
> -Thomas Edwards

Agreed, but I was concentrating on the richness of interconnection; I
neglected to mention that the synapse itself is one of the "hookup" points
for these cells. Although it's often possible to redraw a complex circuit
in a simpler fashion by "lumping" elements to create locally more involved
transfer functions, this generally obscures the simpler structure (_vide_
feedback-type circuits). The astrocytes, being ubiquitous and highly-
connected, explode the parallelism even more than was thought - and that   
must have an impact, at some level, on future modeling with fidelity.
Andrew Palfreyman	nsc!logic!andrew