Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!tut.cis.ohio-state.edu!ucbvax!bloom-beacon!eru!hagbard!sunic!dynas!neural!egel
From: egel@neural.dynas.se (Peter Egelberg)
Newsgroups: comp.ai.neural-nets
Subject: Re: Back-Propagation Weight Initialization
Message-ID: <1990Dec11.091222.3501@neural.dynas.se>
Date: 11 Dec 90 09:12:22 GMT
References: <1325@helens.Stanford.EDU>
Organization: Neural AB, Lund, SWEDEN
Lines: 30

In article <1325@helens.Stanford.EDU> news@helens.Stanford.EDU (news) writes:

>I have a paper titled "Improving the learning speed of two-layer
>networks by choosing the initial values of the adaptive weights" in
>the proceedings of the IJCNN, June 1990, San Diego, page III-21.
> .
> .
> .
>(I get a factor of 4 or 5 improvement in learning time over random
>initial weights.)

The improvement in learning speed sounds fine. But what about generalization.
Does weight initialization improve generalization?

In most applications learning time is not a major problem, the end user
is not going to train the network. I don't mind waiting if I know that I'll
get a network that solves my problem. Generally I think there is too much
focus on learning speeds. When neural networks move to hardware learning
speed is not going to be a problem. But generalization will still be a problem!
I'm not saying that learning speed is unimportant. I'm saying that
generalization is a greater problem when using neural networks in real world
applications.

Thanks,
Peter Egelberg
-- 
Peter Egelberg			E-mail:	egel@neural.dynas.se
Neural AB			Phone:	+46 46 11 00 90
Otto Lindbladsv. 5
223 65 LUND, SWEDEN