Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sun-barr!apple!agate!shelby!portia.stanford.edu!portia!bellido From: bellido@aragorn.world (Ignacio Bellido) Newsgroups: comp.ai.neural-nets Subject: Re: Backprop Weight Initialization Message-ID: Date: 6 Dec 90 21:56:49 GMT References: <16266@imag.imag.fr> <1990Dec6.161422.5314@cs.utk.edu> Sender: news@portia.Stanford.EDU Reply-To: bellido@psych.stanford.edu Organization: /user/bellido/.organization Lines: 32 In-Reply-To: kolen-j@retina.cis.ohio-state.edu's message of 6 Dec 90 20:25:48 GMT I saw that poster also in last NIPS, but I don't believe this is very important, I think all of us who work with backpropagation have these experience and have noted this defect (yes, it's not just a virtue). My own simulator keeps track of the random seed used to initialize each process. What I think is really important is to find some way on wich backpropagation can be made almost independent of this initial configuration and reduce the number of epochs to the least possible. This can be done, I have found a way to reduce the number of epochs by changing the learning rate and momentum factor (other two variables that changes the network behavior). How final weights are placed is not really important if the network realizes the function it has been trained to. This is only important if you want to extract knowledge of the network and this is really difficult with backpropagation. Ignacio Bellido -- -------------------------------------------------------------------------- Ignacio Bellido Fernandez-Montes -1z Visiting Scholar at Stanford University e-mail: bellido@psych.stanford.edu Psychology Department Graduate Student Madrid University of Technology Department of Telematic Engineering e-mail: ibellido@dit.upm.es --------------------------------------------------------------------------