Xref: utzoo sci.math:11723 comp.theory.dynamic-sys:78
Path: utzoo!attcan!uunet!tut.cis.ohio-state.edu!neuron.cis.ohio-state.edu!kolen-j
From: kolen-j@neuron.cis.ohio-state.edu (john kolen)
Newsgroups: sci.math,comp.theory.dynamic-sys
Subject: Re: Dynamical Systems Question
Message-ID: <KOLEN-J.90Jul17095805@neuron.cis.ohio-state.edu>
Date: 17 Jul 90 13:58:05 GMT
References: <1990Jul16.140630.10110@usenet.ins.cwru.edu>
Sender: news@tut.cis.ohio-state.edu
Followup-To: sci.math
Organization: Ohio State Computer Science
Lines: 107
In-reply-to: beer@cthulhu.ces.cwru.edu's message of 16 Jul 90 14:06:30 GMT

In article <1990Jul16.140630.10110@usenet.ins.cwru.edu> beer@cthulhu.ces.cwru.edu (Randall D. Beer) writes:

   From: beer@cthulhu.ces.cwru.edu (Randall D. Beer)
   Newsgroups: sci.math,comp.theory.dynamic-sys
   Date: 16 Jul 90 14:06:30 GMT
   Sender: news@usenet.ins.cwru.edu
   Organization: Computer Engineering and Science/CWRU

   Suppose we are given a system of N differential equations of the form 

			  N
   Ci * dyi/dt = -yi + F(sum{Wji * yi})     where F(x) = 1/(1 + e^(-x))
			 j=1

   My question is this: Can this N dimensional dynamical system be made
   to approximate the dynamics of a given M dimensional dynamical system
   (M <= N) on an M dimensional subspace arbitrarily well by appropriate
   adjustment of the time constants (Ci) and coupling strengths (Wij) as
   N -> infinity?  By "approximate the dynamics" I mean that all
   trajectories on the M dimensional subspace of the N dimensional system
   can be made arbitrarily close to those of the system we are
   approximating.  One way of expressing this is that the following term
   should approach 0 for all initial conditions:

		    M   infinity
		   sum{integral{(yi - di)^2 dt}}
		   i=1     0

   where di(t) is the value of the ith state variable of the system we
   are approximating at time t and yi(t) is the value of the
   corresponding state variable in our approximation.

   What I have in mind here is a dynamical analogue to function
   approximation.  If the answer to my question is yes, can any bounds be
   placed on how the accuracy of the approximation will scale with N?  If
   the answer is no, do any such "basis systems" exist?  Can the above
   system at least approximate an arbitrary M dimensional attractor,
   rather than the entire phase space?

   Any help on these questions would be greatly appreciated.

   Thanks, 

   R. Beer 
   (beer@cthulhu.ces.cwru.edu)

The answer to your question is no.  The approximation dynamic described
above is limited to convex transition functions.  If the dynamic is modified
as follows

			  N2          N1
   Ci * dyi/dt = -yi + F(sum(Vij * F(sum{Wjk * yk}))),
			 j=1         k=1

then any dynamical system with an underlying dynamic expressable as
a Borel measurable function can be approximated within any epsilon
with finite N1 and N2.  Hornik, Stinchcombe, and White proved that any
function from that class can be approximated by a system of functions of
this form

The system you are describing is known in some circles as a recurrent neural
network.  These networks have been studied for DS with fixed point behavior
(Pineada), limit cycle behavoir (Pearlmutter), finite state machines (Pollack),
chaotic systems (Weigend, Huberman,Rumelhart).  This list is by no means
complete, but should give you a good head start.

%A K. Hornik
%A M. Stinchcombe
%A H. White
%T Multi-layer Feedforward Networks are Universal Approximators
%J Neural Networks
(This has appeared but I don't have any more info on it)

%A F. J. Pineda
%T Generalization of Back-Propagation to Recurrent Neural Networks
%J Physical Review Letters
%D 1987
%V 59
%P 2229-2232

%A B. A. Pearlmutter
%T Learning State Space Tragectories in Recurrent Neural Networks
%J Neural Computation
%V 1
%P 263-269
%D 1989

%A J. B. Pollack
%T The Induction of Dynamical Recognizers
%R Tech Report 90-JP-Automata
%I LAIR, Ohio State University
%C Columbus, OH 43210
%D 1990

%A A. S. Weigend
%A B. A. Huberman
%A D. E. Rumelhart
%T Predicting the Future:  A Connectionist Approach
%R Tech Report Stanford-PDP-90-01
%I Stanford Univeristy
%D 1990

--
John Kolen (kolen-j@cis.ohio-state.edu)|computer science - n. A field of study
Laboratory for AI Research             |somewhere between numerology and
The Ohio State Univeristy	       |astrology, lacking the formalism of the
Columbus, Ohio	43210	(USA)	       |former and the popularity of the latter