Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!uwm.edu!bionet!agate!shelby!portia.stanford.edu!whirlwind.Stanford.EDU!lehr
From: lehr@whirlwind.Stanford.EDU (Mike Lehr)
Newsgroups: comp.ai.neural-nets
Subject: Re: Backpropagation with Newton's Method, and recurrence.  Source code.
Message-ID: <1990Nov29.050147.4874@portia.Stanford.EDU>
Date: 29 Nov 90 05:01:47 GMT
References: <7684@uwm.edu> <3146@exodus.Eng.Sun.COM> <7937@uwm.edu>
Sender: news@portia.Stanford.EDU (USENET News System)
Organization: Stanford University EE Dept.
Lines: 22

In article <7937@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes:
>In article <3146@exodus.Eng.Sun.COM> landman@hanami.Eng.Sun.COM (Howard A. Landman) writes:
>>>In article <1248@helens.Stanford.EDU> wan@isl.Stanford.EDU (Eric A. Wan) writes:
>>>>You mention the method does not work well for f(x) = x^2.  In fact, Newton's
>>>>Method applied to the gradient of x^2 converges in one step.
>>
>>In article <7684@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes:
>>>You lost me here.  Try using Newton's method to find a zero of x^2.
>>
>>The gradient of x^2 is 2x.
>
>Hence the iteration formula:
>
>	      x[n+1] = x[n] - f(x[n])/f'(x[n]) = 1/2*x[n].
>
>So Newton's Method applied to x^2 converges only as slowly as the Method of
>Bisection.  The original article wasn't talking about using Netwon's Method
>on the gradient of x^2, but on x^2.

We all give up.

-M. Lehr, F. Beaufays, E. Wan, S. Piche