Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!crdgw1!greenba From: greenba@gambia.crd.ge.com (ben a green) Newsgroups: comp.ai.neural-nets Subject: Re: generalization in NN's Message-ID: Date: 3 Apr 91 18:12:10 GMT References: <1991Apr2.205240.24668@milton.u.washington.edu> <1991Apr3.172206.23469@ux1.cso.uiuc.edu> Sender: news@crdgw1.crd.ge.com Organization: GE Corporate Research & Development Lines: 34 In-reply-to: andy@honda.ece.uiuc.edu's message of 3 Apr 91 17:22:06 GMT In article <1991Apr3.172206.23469@ux1.cso.uiuc.edu> andy@honda.ece.uiuc.edu (Andy Bereson) writes: > > I have a problem with the ability of a neural net to generalize. > > When I use a simple linear discriminant function with seperate > covariance matrices and compare that against a NN with 6 input, > 12 hidden and 4 output nodes. Here's what I get for correct > > LDF NN > train 48.5 59.0 > test 42.0 37.0 > A linear discriminant function is similar to back-prop with no hidden units, and that does better than your twelve hidden units. It sounds like you may be using too many units. This is a common cause for under- generalization. Twelve hidden units can learn two-to-the-twelfth subclasses if one trains to perfection. The original poster had only 400 training patterns, which is less than two-to-the-ninth. So I heartily agree with Andy that one should use many fewer hidden nodes in the quoted problem. Try 2 for starters. Ben -- Ben A. Green, Jr. greenba@crd.ge.com Speaking only for myself, of course.