Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!iuvax!rutgers!psuvax1!news
From: news@psuvax1.cs.psu.edu (The Usenet)
Newsgroups: comp.ai.neural-nets
Subject: Re: NN Question
Message-ID: <8903071701.AA12290@shire.cs.psu.edu>
Date: 7 Mar 89 17:01:17 GMT
References: <32125@gt-cmmsr.GATECH.EDU>
Organization: Penn State University
Lines: 61
To: sim!brp
In-Reply-To: <10624@pasteur.Berkeley.EDU>

In article <10624@pasteur.Berkeley.EDU> sim!brp writes:

> I think that a great many people view neural networks as good
> models for what goes on inside our heads.  Since these models
> are, mainly, discrete time automata they do not reflect the
> fact that real neural systems are, essentially, nonlinear
> continuous-time multi-dimensional vector spaces in which
> the neurons evolve in time.  So while they are real neat
> computational tools, they are far from representing real
> neural processes.

I think you are guilty of over-stating the case for your discipline.
Real neural systems are real neural systems.  They are not "nonlinear
continuous-time multi-dimensional vector spaces", although it may be
constructive to model them as such.

Real neural systems can also be modelled as (borrowing your terminology)
"discrete time automata".  One must distinguish between reality and
the scientific model of choice.  I believe that you meant to say that
modelling real neural systems as "nonlinear continuous-time multi-
dimensional vector spaces" leads to a better understanding of real
neural systems than modelling them as "discrete time automata".

The discrete vs continuous competition is not new.  You sit on the
same side of the fence as many distinguished people.  I lean towards
the discrete side myself, although I am open to argument.

I have not seen any arguments which convince me that the analog
behaviour that we observe in real neural systems is of fundamental
computational importance.  Some of the arguments that I have seen
have been based on the premise that the real world is analog.
Unfortunately, the real world appears to be discrete.  By this I mean
that scientific models which are based on discrete units (atoms,
quarks etc.) give a good understanding of observable phenomena.
Real numbers, continuous functions etc., are abstractions which help
us deal with the fact that the number of discrete units is larger
than we can deal with comfortably.

There are (at least) two objections to the classical automata-
theoretic view of neural systems.  One is that neural systems
are not clocked (I presume that this is what you mean by
"continuous time"), and that neurons have analog behaviour.
Two burning questions which, in my mind, are among the
most important open questions in neural networks research are:
1.  Is unclocked behaviour important?  Was the non-availability
    of a system clock something that Nature had to fight to overcome,
    or did it bring inherent advantages?
2.  Is analog behaviour important?  If I restrict neuron excitation
    values to 6 decimal places, will the networks still function
    correctly?  More importantly, how does the precision scale with
    the number of neurons and/or connections?

Needless to say, these questions are not new.  I am not claiming to
be the first person to have thought of them.  Some information is known.
I am  planning two papers this year (not yet written up) which address
aspects of them.  The Truth (if it exists) still remains to be found.
-------------------------------------------------------------------------------
			Ian Parberry
  "The bureaucracy is expanding to meet the needs of an expanding bureaucracy"
  ian@psuvax1.cs.psu.edu  ian@psuvax1.BITNET  ian@psuvax1.UUCP  (814) 863-3600
 Dept of Comp Sci, 333 Whitmore Lab, Penn State Univ, University Park, Pa 16802