Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!think.com!zaphod.mps.ohio-state.edu!cis.ohio-state.edu!tut.cis.ohio-state.edu!sei.cmu.edu!fs7.ece.cmu.edu!o.gp.cs.cmu.edu!andrew.cmu.edu!rr2p+ From: rr2p+@andrew.cmu.edu (Richard Dale Romero) Newsgroups: comp.ai.neural-nets Subject: Re: Adding 0.1 to logistic Message-ID: Date: 23 May 91 22:54:06 GMT References: <{HV_2H$@warwick.ac.uk> Organization: Carnegie Mellon, Pittsburgh, PA Lines: 18 In-Reply-To: <{HV_2H$@warwick.ac.uk> in response to patrick's statement about pushing the logistic towards a more non-linear section, i think he was slightly off about what the .1 was being added to. it would make more sense to add .1 to the output of the logistic, not it's input. the bias parameter takes care of any movements along the logistic curve that you need to make. a possible reason as to why adding this .1 would speed up learning has been brought up before on this group, i believe, or something along those lines. the two things i do remember are subtracting .5 from the logistic so that it is centered around 0, or adding 0.1 to the sigmoid prime function. both are talked about in fahlman's 'empirical study of learning speed in back- propagation networks', cmu-cs-88-162. the reason for adding .1 to the sig- prime function is to avoid letting it go to 0 when the input is at an extreme. the symmetric sigmoid is talked about in stometta and huberman's 'an improved three-layer back-prop algorithm' in proceedings of the ieee international conference on neural networks, pages 637-644, 1987. -rick