Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!swrinde!zaphod.mps.ohio-state.edu!uakari.primate.wisc.edu!aplcen!jhunix!ins_atge
From: ins_atge@jhunix.HCF.JHU.EDU (Thomas G Edwards)
Newsgroups: comp.ai.neural-nets
Subject: Re: Using Newton's Method to speed up backpropagation.
Summary: Need Quickprop
Message-ID: <6905@jhunix.HCF.JHU.EDU>
Date: 17 Nov 90 05:08:18 GMT
References: <2124@cybaswan.UUCP> <6873@jhunix.HCF.JHU.EDU> <86170@lll-winken.LLNL.GOV>
Organization: The Johns Hopkins University - HCF
Lines: 34

In article <86170@lll-winken.LLNL.GOV> loren@dweasel.llnl.gov (Loren Petrich) writes:
>	Does anyone know of any readily-available Cascade-Correlation
>source?

PostScript Paper: ftp from cheops.cis.ohio-state.edu
Source: in C and Lisp from pt.cs.cmu.edu (I think the directory is
   connect/code)

>	I tried writing a Cascade-Correlation routine, without very
>much success.

Did you use Quickprop to train the weights?  Did you use multiple 
candidate units?  I think both are needed for good performance.
I must admit I think Quickprop is kind of a hack, but it does work,
though I think conjugate-gradient methods might also be useful for
training the weights of a cascade-correlation network.

>	I think that the problem of local minima will necessarily
>plague ANY localized search algorithm, because if it finds one
>minimum, it will know nothing about the other minima.
  
  Yeah.  The important thing about cascade-corrletion is that it is
significantly different than backprop since it does not strictly
follow the gradient-descent of error.  It is a kind of "compositional
learning" which creates useful subgoals (i.e. hidden units as feature
detectors which maximally help reduce the error) and puts them together
to solve a larger goal.  Thus it could be scooting all over the
error surface.  Whether or not this reduces local minima problems
is not mathematically obvious, but common-sense and my examination
of results so far says that it probably does (at least for the
two-spirals problem).

-Thomas Edwards
 (Looking for Ph.D. programs to do neurally-inspired VLSI for next fall)