Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sun-barr!lll-winken!dweasel!loren
From: loren@dweasel.llnl.gov (Loren Petrich)
Newsgroups: comp.ai.neural-nets
Subject: Re: Using Newton's Method to speed up backpropagation.
Message-ID: <86351@lll-winken.LLNL.GOV>
Date: 18 Nov 90 23:59:04 GMT
References: <6873@jhunix.HCF.JHU.EDU> <86170@lll-winken.LLNL.GOV> <6905@jhunix.HCF.JHU.EDU>
Sender: usenet@lll-winken.LLNL.GOV
Organization: Lawrence Livermore National Laboratory
Lines: 44
Nntp-Posting-Host: dweasel.llnl.gov

In article <6905@jhunix.HCF.JHU.EDU> ins_atge@jhunix.HCF.JHU.EDU (Thomas G Edwards) writes:
>In article <86170@lll-winken.LLNL.GOV> loren@dweasel.llnl.gov (Loren Petrich) writes:
>>	Does anyone know of any readily-available Cascade-Correlation
>>source?
>
>PostScript Paper: ftp from cheops.cis.ohio-state.edu
>Source: in C and Lisp from pt.cs.cmu.edu (I think the directory is
>   connect/code)

	I've already "been" to cheops.cis.ohio-state.edu, and I had
already gotten Fahlman's paper from there. It is implementing what he
had suggested that was the big trouble.

	I looked in pt.cs.cmu.edu and I was not allowed access to the
contents of connect/code. I got a message stating that

530 Access not allowed for guest users for path connect/code

	I was also not allowed access into any other (suspected) directory.

>>	I tried writing a Cascade-Correlation routine, without very
>>much success.
>
>Did you use Quickprop to train the weights?  Did you use multiple 
>candidate units?  I think both are needed for good performance.
>I must admit I think Quickprop is kind of a hack, but it does work,
>though I think conjugate-gradient methods might also be useful for
>training the weights of a cascade-correlation network.

	I had used Quickprop -- and multiple candidate units. Without
much success. The NN almost always gets stuck with some weights VERY
big and others small. And when I put in a big weight decay rate, the
NN usually gets very lousy results.

	I'm training on 0 or 1 (-1 or 1 in my code) decision problems
like the encoder and line-symmetry problems.


$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
Loren Petrich, the Master Blaster: loren@sunlight.llnl.gov

Since this nodename is not widely known, you may have to try:

loren%sunlight.llnl.gov@star.stanford.edu