Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!bloom-beacon!apple!vsi1!hal
From: hal@vicom.COM (Hal Hardenbergh)
Newsgroups: comp.ai.neural-nets
Subject: Re: Training
Message-ID: <1577@vicom.COM>
Date: 21 Mar 89 21:14:47 GMT
References: <2698@sun.soe.clarkson.edu> <2351@buengc.BU.EDU>
Reply-To: hal@vicom.COM (Hal Hardenbergh (236))
Organization: Vicom Systems Inc. San Jose, Cal.
Lines: 37

In article <2351@buengc.BU.EDU> bph@buengc.bu.edu (Blair P. Houghton) writes:
>In article <2698@sun.soe.clarkson.edu> spam@clutx.clarkson.edu writes:
>>Has anyone come up with a good way to train a net
>>without knowing a "target" in advance?
>[...]
>>All the learning methods I've studied so far
>>are inadequate for this.
>
>Sounds like good ol' negative reinforcement to me.
>
>You could send it to a Zen Buddhist monastery, or an English public
>school...
>
>Have you tried just getting a degree in education and coding it?
>
>				--Blair
>				  "...using C--, the 'Object-Lesson' C..."
>
>P.S.  8-D

A colleague and I have tried several of the back-prop "speedup" methods.  All
that we have tried do speed up convergence, to some degree, as determined by
the number of epochs (training iterations).  However, none of them reliably
provide a speed improvement as measured by wall clock time.

The ones which do (sometimes) provide a slight improvement in wall clock time
do not do so reliably.  It's sort of like varying the convergence and momentum
factors.  Depending on the random initialization of the weights and biases,
c1 and m1 will work better than c2 and m2, and vice versa.

As long as we are simulating artificial neural nets in software (if simulating
is the right word here), does anyone know of a back-prop speedup trick which
reduces the wall-clock training time?

Hal Hardenbergh  [incl std dsclmr]  hal@vicom.com
Vicom Systems Inc  2520 Junction Ave  San Jose  CA  (408) 432-8660
surely nobody here remembers the newsletter DTACK Grounded ?