Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!ucbvax!pasteur!sim!brp
From: brp@sim.uucp (bruce raoul parnas)
Newsgroups: comp.ai.neural-nets
Subject: Re: NN Question
Message-ID: <11114@pasteur.Berkeley.EDU>
Date: 15 Mar 89 17:28:45 GMT
References: <32125@gt-cmmsr.GATECH.EDU> <8903071701.AA12290@shire.cs.psu.edu>
Sender: news@pasteur.Berkeley.EDU
Reply-To: brp@sim.UUCP (bruce raoul parnas)
Organization: University of California, Berkeley
Lines: 89

In article <8903071701.AA12290@shire.cs.psu.edu> news@psuvax1.cs.psu.edu (The Usenet) writes:
>In article <10624@pasteur.Berkeley.EDU> sim!brp writes:
>
>> I think that a great many people view neural networks as good
>> models for what goes on inside our heads.  Since these models
>> are, mainly, discrete time automata they do not reflect the
>> fact that real neural systems are, essentially, nonlinear
>> continuous-time multi-dimensional vector spaces in which
>> the neurons evolve in time.  So while they are real neat
>> computational tools, they are far from representing real
>> neural processes.
>
>I think you are guilty of over-stating the case for your discipline.
>Real neural systems are real neural systems.  They are not "nonlinear
>continuous-time multi-dimensional vector spaces", although it may be
>constructive to model them as such.

actually my discipline is more neurobiology than it is nonlinear systems, 
although i do think they are a good model.  you are right, though, that this
is only a model.  what i meant to say was that i believed that this was a
better modelling approach than automata theory.

>I have not seen any arguments which convince me that the analog
>behaviour that we observe in real neural systems is of fundamental
>computational importance.  Some of the arguments that I have seen
>have been based on the premise that the real world is analog.
>Unfortunately, the real world appears to be discrete.  By this I mean
>that scientific models which are based on discrete units (atoms,
>quarks etc.) give a good understanding of observable phenomena.

the world is (possibly) discrete on a very fine level.  first, it seems to me
that researchers keep finding yet smaller particles into which matter is sub-
divided: maybe it really is a continuum?  second, even assuming that it is
discrete, this exists on such a fine level that i believe it is irrelevant
here.  modelling of neural systems in terms of their atomic properties is, i
believe, quiet the unenviable task!

>Real numbers, continuous functions etc., are abstractions which help
>us deal with the fact that the number of discrete units is larger
>than we can deal with comfortably.

right.  and in most physical systems we may, for our understanding, treat them
as essentially analog since we simply can't deal with the complexity presented
by the true (?) discrete nature.

>There are (at least) two objections to the classical automata-
>theoretic view of neural systems.  One is that neural systems
>are not clocked (I presume that this is what you mean by
>"continuous time"), and that neurons have analog behaviour.

that is precisely what i meant.  neurons each evolve on their own, independent
of system clocks.

>Two burning questions which, in my mind, are among the
>most important open questions in neural networks research are:
>1.  Is unclocked behaviour important?  Was the non-availability
>    of a system clock something that Nature had to fight to overcome,
>    or did it bring inherent advantages?

i believe that a system clock would be more of a hindrance that a help.  
studies with central pattern generators and pacemaker activity (re: the heart)
show clearly that system clocks are not unavailable.  if evolution had found
a neural system clock advantageous, one could have been created.  i feel,
however, that the continuous-time evolution of neural systems imbues them
with their remarkable properties.

>2.  Is analog behaviour important?  If I restrict neuron excitation
>    values to 6 decimal places, will the networks still function
>    correctly?  More importantly, how does the precision scale with
>    the number of neurons and/or connections?

I don't think that such a fine level of precision is necessary in neural
function, i.e. six places would likely be enough.  but since digital circuitry
is made actaully from analog circuit elements limited to certain regions of
operation, why go to this trouble in real neural systems when analog seems
to work just fine?
>
>Needless to say, these questions are not new.  I am not claiming to
>be the first person to have thought of them.  Some information is known.

>I am  planning two papers this year (not yet written up) which address
>aspects of them.  The Truth (if it exists) still remains to be found.

I would be very interested in getting preprints of this work when it becomes
available.  i, too, am open to arguement for my views.


bruce
brp@sim