Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!swrinde!elroy.jpl.nasa.gov!ncar!gatech!utkcs2!mars!sfp
From: sfp@mars.ornl.gov (Phil Spelt)
Newsgroups: comp.ai.neural-nets
Subject: Re: The first few epochs in BP
Summary: Chaotic behavior in BP nets
Message-ID: <1991Apr3.183249.7767@cs.utk.edu>
Date: 3 Apr 91 18:32:49 GMT
References: <6882@rex.cs.tulane.edu>
Sender: Phil Spelt
Reply-To: sfp@mars.epm.ornl.gov (Phil Spelt)
Organization: Oak Ridge National Lab
Lines: 35

In article <6882@rex.cs.tulane.edu> georgiou@rex.cs.tulane.edu (George Georgiou) writes:
>For those who worked with Back-Propagation: Have you notice any
>chaotic behavior in the graph of the (usual) error function vs epochs?
>Specifically, during the first 2 of 3 epochs the value of the error
>would jump all over the place, but afterwords becomes smooth.
>
>Only once I saw this behavior in the literature. It was in a graph in
>a paper in a respected publication, but it was ignored.
>
>Is this symptomatic of gradient descent procedures?
>
>George Georgiou                       georgiou@rex.cs.tulane.edu
>Computer Science Department           +---------------------------+
>Tulane University                     |       Fiat Lux            |
>New Orleans, LA 70118                 +---------------------------+

I cite the following:

"Chaos and the Step-Size Dilemma in the Back-Prop Learning Algorithm"  by
R. Chris Lacher and Michael E. Manausa, Dept Comp. Sci., Fla. State Univ.
in:  Proceedings of the Second Workshop on Nerual Networks:  Academic/
Industrial/Defense,  WNN-AIND 91;  held at Auburn University, AL, 11-13 Feb, 1991.

It makes for VERY interesting reading!
=============================================================================
MIND.  A mysterious form of matter secreted by the brain.  Its chief activity
consists in the endeavor to asscertain its own nature, the futility of the
attempt being due to the fact that it has nothing but itself to know itself
with.   -- Ambrose Bierce
=============================================================================

Phil Spelt, Cognitive Systems & Human Factors Group  sfp@epm.ornl.gov
============================================================================
Any opinions expressed or implied are my own, IF I choose to own up to them.
============================================================================