Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!gem.mps.ohio-state.edu!apple!motcsd!hpda!hpcuhc!edwardm
From: edwardm@hpcuhc.HP.COM (Edward McClanahan)
Newsgroups: comp.ai.neural-nets
Subject: Re: PDP Question
Message-ID: <2240002@hpcuhc.HP.COM>
Date: 20 Sep 89 01:40:37 GMT
References: <20900@uflorida.cis.ufl.EDU>
Organization: Hewlett Packard, Cupertino
Lines: 31

Tom Krajna writes:

> I'm using version 1.1 of the PDP software.  When using the backprop
> program (bp), the program runs until the total sum of squares (tss)
> drops below a critical error level (ecrit).  At this point, if a
> "test all" command (tall) is executed, and you test all the patterns,
> the tss is no longer the same as the one which had just caused the
> program to stop running.  (As a matter of fact, it seems to be the
> "next" tss that would be calculated, if you ran all the patterns through
> one more time, and adjusted the weights.)

> Why is this a problem?  Sometimes, due to the meanderings of the
> gradient descent, the "next" tss may be greater than the one which
> was just low enough to meet the ecrit criterion.  So your net has
> stopped, but statistically, it may not be a good enough fit.

> Anyone have a solution or a work-around?

I don't know about a "solution" or a "work-around", but this sounds
like you have found a "local minima".  Sometimes the steepest "decent"
isn't always the best.

Ed "neophite at PDP" McClanahan

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

  Edward McClanahan
  Hewlett Packard Company
  Mail Stop 47UE              -or-     edwardm%hpda@hplabs.hp.com
  19447 Pruneridge Avenue
  Cupertino, CA  95014                 Phone: (408)447-5651