Path: utzoo!attcan!uunet!midway!ncar!elroy.jpl.nasa.gov!usc!zaphod.mps.ohio-state.edu!uakari.primate.wisc.edu!aplcen!boingo.med.jhu.edu!haven!cs.wvu.wvnet.edu!cerc.wvu.wvnet.edu!babcock.cerc.wvu.wvnet.edu
From: clchen@babcock.cerc.wvu.wvnet.edu (Chih-Liang Chen)
Newsgroups: comp.ai.neural-nets
Subject: Re: PDP's backprop program - Limit on number of units?
Message-ID: <1039@babcock.cerc.wvu.wvnet.edu>
Date: 1 Dec 90 06:46:23 GMT
References: <2370@sparko.gwu.edu>
Sender: news@cerc.wvu.wvnet.edu
Distribution: na
Lines: 30

From article <2370@sparko.gwu.edu>, by fc174103@seas.gwu.edu (Burr):
> I'm using the backpropagation program that comes with the PDP books.
> I'm trying to train a network with 45 units (20 input, 15 hidden, and
> 10 output), to recognize the digits 0 thru 9.  I've set up a template
> file that uses 132 columns by 43 lines to display appropriate info
> about the network while it is being trained.  The display works fine
> and does not seem to be the thing which is hanging the network up.  The
> network hangs when I try to train it using the ptrain or strain commands.
> It even hangs when I use the reset command.  While these commands do not
> work the tall command, which just computes activations without changing
> the weights, does work.  I'm not sure what is the problem.  Is 45 units
> pushing the limit of this backprop program?  If anyone has had experience
> with this program I would much appreciate any help you could offer.

It does has this problem, but it does not mean the BP algorithm doesn't
work.  My experience shows that:

	1. BP is good learning algorithm
	2. training by epoch is the worst for large training patterns
	3. strain is            the 2nd .........
	4. ptrain, you can take chance.

Your NN is a "small" one, I had 256X64X4 with 512 training patterns. It got
stuck with "epoch" or "pattern" method.  I solved it by
"intensive training on the pattern which makes the maximum MS error".  So,
you can:
	1. modify the bp program, or
	2. write your own one.

just a experience.