Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!swrinde!elroy.jpl.nasa.gov!lll-winken!tristan!loren
From: loren@tristan.llnl.gov (Loren Petrich)
Newsgroups: comp.ai.neural-nets
Subject: Anybody's Experience with Fahlman's Quickprop? (was Re: Are Conjugate Gradient algorithms any good?)
Keywords: Conjugate Gradient algorithms, Back-propagation
Message-ID: <92992@lll-winken.LLNL.GOV>
Date: 11 Mar 91 22:10:54 GMT
References: <47034@nigel.ee.udel.edu>
Sender: usenet@lll-winken.LLNL.GOV
Organization: Lawrence Livermore National Laboratory
Lines: 32
Nntp-Posting-Host: tristan.llnl.gov


	Having reviewed some Conjugate Gradient methods, I find
them rather complicated.

	An alternative, due to Fahlman, is the Quickprop algorithm. It
is described in some papers of his that can be found in the
/pub/neuroprose directory of cheops.cis.ohio-state.edu, available by
anonymous ftp.

	Basically, it works by remembering the previous gradient and
the stepsize taken from there, and finding the new weight values by
fitting a line from the current gradient to the previous gradient.
This operation is done on each weight component separately. In effect,
the Hessian is approximated as a diagonal matrix, but one where the
nonzero elements are independent of each other. There are some fudge
factors that have to be added here and there, such as adding a
gradient-descent "starter" and keeping the stepsizes from growing too
rapidly, but this algorithm is remarkably simple.

	I have found it to be a stable and fast algorithm for solving
gradient-descent problems.

	Has anyone else had experience with Quickprop, and how does it
compare with Conjugate Gradients and other such methods?


$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
Loren Petrich, the Master Blaster: loren@sunlight.llnl.gov

Since this nodename is not widely known, you may have to try:

loren%sunlight.llnl.gov@star.stanford.edu