Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!purdue!mentor.cc.purdue.edu!j.cc.purdue.edu!pur-ee!pc.ecn.purdue.edu!cb.ecn.purdue.edu!kavuri
From: kavuri@cb.ecn.purdue.edu (Surya N Kavuri )
Newsgroups: comp.ai.neural-nets
Subject: Re: Training
Message-ID: <774@cb.ecn.purdue.edu>
Date: 24 Mar 89 05:31:02 GMT
References: <2698@sun.soe.clarkson.edu> <2351@buengc.BU.EDU> <1577@vicom.COM> <7326@phoenix.Princeton.EDU>
Organization: Purdue University Engineering Computer Network
Lines: 10


 Besides gradient and conjugate gradient methods there are other one could try.
 There are methods known as Quasi-Newton methods that are known
 to perform much better in non-linear optimization.  It is 
 because they use higher order derivatives besides the first(as  in gradient methods), and thus use more knowledge of the 
 objective function contours.  Second and higher order derivatives can be thought of as indicators of error (surfaces) arising from gradient application.  This additional knowledge is used to achieve a much rapid convergence.  
 The computational difficulties with quasi-Newtonian approaches is the evaluation of the Hessian.
 There are inexpensive updating methods as BFGS(Broyden-Fletcher -Goldfarb-Shannon) algorithm.
      (most NLP books should have this)
                                           Surya