Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!mips!atha!aunro!alberta!arms From: arms@cs.UAlberta.CA (Bill Armstrong) Newsgroups: comp.ai.neural-nets Subject: Re: Backprop with additional noisy inputfeature Message-ID: Date: 15 May 91 23:51:56 GMT References: <3559@wn1.sci.kun.nl> <1991May14.182932.21193@convex.com> <15347@arctic.nprdc.navy.mil> Sender: news@cs.UAlberta.CA (News Administrator) Organization: University of Alberta, Edmonton, Canada Lines: 29 In article <3559@wn1.sci.kun.nl>, mderksen@sci.kun.nl (M. Derksen) writes: > Dear neural netters, > > Can someone explain me why an additional noisy inputfeature results in a > better prediction performance (generalization). > I think the effect of adding noise to the input variables is really not an effect on the learning system, but is rather just changing the training set. The new one has some points in it obtained by "extrapolating" from a given input vector to one near it (the "noisy" one). The original vector and the noisy one are given the same output. From there, one can modify the procedure to be even more general: look at small groups of training points in close proximity and generate a new vector by linear regression, for example. One could also perform a K-nearest-neighbors (KNN) classification and use that to train the network. The above all generate new, better training sets. In the case of using KNN, the work of creating a good decision boundary is thus removed from the network, which just has to learn the training data well. The network is still useful in feedforward mode because of its high speed. -- *************************************************** Prof. William W. Armstrong, Computing Science Dept. University of Alberta; Edmonton, Alberta, Canada T6G 2H1 arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071