Xref: utzoo sci.electronics:3557 comp.ai:2098 comp.ai.neural-nets:181
Path: utzoo!attcan!uunet!husc6!rutgers!ucsd!ucbvax!icsia!munro
From: munro@icsia.berkeley.edu (Paul Munro)
Newsgroups: sci.electronics,comp.ai,comp.ai.neural-nets
Subject: Re: Sigmoid transfer function
Message-ID: <25516@ucbvax.BERKELEY.EDU>
Date: 7 Aug 88 19:55:49 GMT
References: <1945@aecom.YU.EDU> <17615@glacier.STANFORD.EDU>
Sender: usenet@ucbvax.BERKELEY.EDU
Reply-To: munro@icsia.UUCP (Paul Munro)
Organization: Postgres Research Group, UC Berkeley
Lines: 24

In article <17615@glacier.STANFORD.EDU> jbn@glacier.UUCP (John B. Nagle) writes:
[JN]Recognize that the transfer function in a neural network threshold unit
[JN]doesn't really have to be a sigmoid function.  It just has to look roughly
[JN]like one.  The behavior of the net is not all that sensitive to the 
[JN]exact form of that function.  It has to be continuous and monotonic, 
[JN]reasonably smooth, and rise rapidly in the middle of the working range.
[JN]The trigonometric form of the transfer function is really just a notational
[JN]convenience.
[JN]
[JN]   It would be a worthwhile exercise to come up with some other forms
[JN]of transfer function with roughly the same graph, but better matched to
[JN]hardware implementation.  How do real neurons do it?
[JN] 
[JN] 					John Nagle


Try this one :   f(x) = x / (1 + |x|)


It is continuous and differentiable:  

f'(x)  =  1 / (1 + |x|) ** 2   =   ( 1 - |f|) ** 2 .

- Paul Munro