Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!wuarchive!rex!georgiou
From: georgiou@rex.cs.tulane.edu (George Georgiou)
Newsgroups: comp.ai.neural-nets
Subject: The first few epochs in BP
Message-ID: <6882@rex.cs.tulane.edu>
Date: 3 Apr 91 00:11:33 GMT
Organization: Computer Science Dept., Tulane Univ., New Orleans, LA
Lines: 25

For those who worked with Back-Propagation: Have you notice any
chaotic behavior in the graph of the (usual) error function vs epochs?
Specifically, during the first 2 of 3 epochs the value of the error
would jump all over the place, but afterwords becomes smooth.

Only once I saw this behavior in the literature. It was in a graph in
a paper in a respected publication, but it was ignored.

Is this symptomatic of gradient descent procedures?

I'll appreciate comments.

-------------------------
1) Alternative title of this posting: The first few moments of the Big
Bang.

2) Seen on a bumper sticker (in reference to the earth and the
environment): "Think globally, act locally." It would make a nice
motto of some Neural Net society.
------------------------

George Georgiou                       georgiou@rex.cs.tulane.edu
Computer Science Department           +---------------------------+
Tulane University                     |       Fiat Lux            |
New Orleans, LA 70118                 +---------------------------+