Path: utzoo!utgpu!news-server.csri.toronto.edu!neuron.ai.toronto.edu!tap
Newsgroups: comp.ai.neural-nets
From: tap@ai.toronto.edu (Tony Plate)
Subject: Re: BP input scaling, normalization
Message-ID: <91Apr22.153535edt.275@neuron.ai.toronto.edu>
Keywords: BP, scaling
Organization: Department of Computer Science, University of Toronto
References: <2533@fornax.UUCP>
Distribution: na
Date: 22 Apr 91 19:35:46 GMT
Lines: 36

In article <2533@fornax.UUCP> mcguire@fornax.UUCP (Michael McGuire) writes:
>
>Questions:
>	1. What are the effects of scaling the inputs to a BP net and is
>	   there an optimal way to do this (especialy since I have 2 sets of
>	   inputs that need to be scaled differently).
Scaling inputs has no effect on what solutions can be implemented
by the network (since the weights can be scaled to compensate), but
it might effect the training.
>	2. Why would a single-layer net outperform a two-layer net (2-layer
>	   net only had 5 hidden units). I would expect the two-layer net to
>	   at least do as well.
Two possible reasons (there may be others)
(1) Not enough hidden units
(2) Net is overtrained

Solutions are:
(1) Increase number of hidden units
(2) Stop training earlier (use cross validation to decide when to stop)

>	3. Do output activations of 0.1 and 0.9 (as opposed to 0.0 and 1.0)
>	   help the generalization process.

This might help, but in my experience it is better to use softmax outputs.

>	4. Is there a different neural net better suited to this type of
>	   classification (Radial Basis Functions?).
>
Depends on the shape of the classes in input space.

-- 
---------------- Tony Plate ----------------------  tap@ai.utoronto.ca -----
Department of Computer Science, University of Toronto, 
10 Kings College Road, Toronto, 
Ontario, CANADA M5S 1A4
----------------------------------------------------------------------------