Path: utzoo!news-server.csri.toronto.edu!rutgers!sun-barr!newstop!exodus!hanami.Eng.Sun.COM!landman From: landman@hanami.Eng.Sun.COM (Howard A. Landman) Newsgroups: comp.ai.neural-nets Subject: Re: Are Conjugate Gradient algorithms any good? Keywords: NETtalk, Conjugate Gradient algorithms, Back-propagation Message-ID: <9682@exodus.Eng.Sun.COM> Date: 12 Mar 91 21:48:20 GMT References: <1991Mar4.142559.21857@daimi.aau.dk> <^9B&5R#@warwick.ac.uk> <91Mar7.145659edt.437@neuron.ai.toronto.edu> Sender: news@exodus.Eng.Sun.COM Organization: Sun Microsystems, Mt. View, Ca. Lines: 27 In article <91Mar7.145659edt.437@neuron.ai.toronto.edu> radford@ai.toronto.edu (Radford Neal) writes: >In any comparison of learning methods, one must be clear on what the cost >criterion is Absolutely. >typically, however, the training data is not all that voluminous, and >omitting some patterns might seriously impair gradient descent unless >the learning rate is set quite low. Do you think it would be fair to say that training data is typically not very large because people simply don't have machines powerful enough (or algorithms efficient enough) to deal with anything larger? I could easily be running a few hundred thousand patterns of a few hundred inputs each into a few thousand neurons, *IF* I had anything that could handle it. One disadvantage of CG methods is that they often require the whole training set to be memory-resident. For gigantic training data this can be a real problem. Does anyone have any insights on methods for handling large amounts of training data efficiently? -- Howard A. Landman landman@eng.sun.com -or- sun!landman