Path: utzoo!mnetor!tmsoft!torsqnt!news-server.csri.toronto.edu!cs.utexas.edu!wuarchive!zaphod.mps.ohio-state.edu!uakari.primate.wisc.edu!aplcen!simonof From: simonof@aplcen.apl.jhu.edu (Simonoff Robert 301 540 1864) Newsgroups: comp.ai.neural-nets Subject: Re: Dynamic range of nodes Message-ID: <1990Dec22.042610.23800@aplcen.apl.jhu.edu> Date: 22 Dec 90 04:26:10 GMT References: <1990Dec21.010536.17034@aplcen.apl.jhu.edu> <8513@uwm.edu> Reply-To: simonof@aplcen (Simonoff Robert 301 540 1864) Organization: Johns Hopkins University Lines: 54 In article <8513@uwm.edu> markh@csd4.csd.uwm.edu (Mark William Hopkins) writes: >In article <1990Dec21.010536.17034@aplcen.apl.jhu.edu> simonof@aplcen.apl.edu (Simonoff Robert 301 540 1864) writes: >>I have choseen as my new activation function the hyperbolic >>tangent function which is defined from [-1.0, 1.0]. The >>derivative of this function is: >> 2 1 >>tanh'(X) == sech (X) == --------- >> 2 >> cosh (X) > >... = 1 - (tanh(x))^2. ... > >(A code fragment was presented with the question "what's wrong with it?") > >Thus, in: >>int compute_delta(int pattern) >... >> delta2[pattern][i] = (target[pattern][i]-out2[pattern][i]) * >> 1.0/(cosh(out2[pattern][i])* >> cosh(out2[pattern][i])); > >should be > delta2[pattern][i] = (target[pattern][i]-out2[pattern][i]) * > (1 - out2[pattern][i]*out2[pattern][i]); > >and > >> delta1[pattern][i] = sum * 1.0/(cosh(out1[pattern][i])* >> cosh(out1[pattern][i])); > >should be > delta1[pattern][i] = sum * (1 - out1[pattern][i]*out1[pattern][i]); Why is delta1[pattern][i] = sum*(1-out1[pattern][i]*out1[pattern][i]) ? My activation function is tanh(netinput) and I believe the derivative of hyperbolic tanget is: tanh'(x) = 1/sech(x)**2 = 1/cosh(x)**2 = 2/(e**x + e**(-x)) Maybe I am not seeing the algebra that makes: 2 1 --------------- = ------------------ + -1 x -x -x -x e + e (1 + e ) (1 + e ) Bob Simonoff simonof@aplcen.apl.edu -- *********************************************************** Bob Simonoff simonof@aplcen Johns Hopkins University