Path: utzoo!mnetor!tmsoft!torsqnt!news-server.csri.toronto.edu!cs.utexas.edu!wuarchive!uwm.edu!csd4.csd.uwm.edu!markh
From: markh@csd4.csd.uwm.edu (Mark William Hopkins)
Newsgroups: comp.ai.neural-nets
Subject: Re: Dynamic range of nodes
Summary: Doing backpropagation on the dynamic range [-1, 1].
Message-ID: <8513@uwm.edu>
Date: 21 Dec 90 17:49:37 GMT
References: <1990Dec21.010536.17034@aplcen.apl.jhu.edu>
Sender: news@uwm.edu
Organization: University of Wisconsin - Milwaukee
Lines: 31

In article <1990Dec21.010536.17034@aplcen.apl.jhu.edu> simonof@aplcen.apl.edu (Simonoff Robert  301 540 1864) writes:
>I have choseen as my new activation function the hyperbolic
>tangent function which is defined from [-1.0, 1.0].  The
>derivative of this function is:
>                2            1
>tanh'(X) == sech (X)  ==  ---------
>                              2
>                          cosh (X)

... = 1 - (tanh(x))^2. ...

(A code fragment was presented with the question "what's wrong with it?")

Thus, in:
>int compute_delta(int pattern)
...
>      delta2[pattern][i] = (target[pattern][i]-out2[pattern][i]) *
>                             1.0/(cosh(out2[pattern][i])*
>                                  cosh(out2[pattern][i]));

should be
       delta2[pattern][i] = (target[pattern][i]-out2[pattern][i]) *
			    (1 - out2[pattern][i]*out2[pattern][i]);

and

>      delta1[pattern][i] = sum * 1.0/(cosh(out1[pattern][i])*
>                              cosh(out1[pattern][i]));

should be
       delta1[pattern][i] = sum * (1 - out1[pattern][i]*out1[pattern][i]);