Path: utzoo!utgpu!attcan!uunet!cs.utexas.edu!tut.cis.ohio-state.edu!ukma!psuvm.bitnet!cunyvm!nyser!cmx!mkciliz
From: mkciliz@cmx.npac.syr.edu (M. Kemal Ciliz)
Newsgroups: comp.ai.neural-nets
Subject: Re: Does back-propagation work with a wider dynamic range...?
Keywords: back propagation, linear output functions...
Message-ID: <1580@cmx.npac.syr.edu>
Date: 24 May 89 04:12:37 GMT
References: <31641@sri-unix.SRI.COM>
Reply-To: mkciliz@cmx.npac.syr.edu,olky@vax5.cit.cornell.edu (M. Kemal Ciliz)
Organization: Northeast Parallel Architectures Center, Syracuse NY
Lines: 24


        Hi again,

        Thanks to all who sent messages to me explaining possible
        cures to my problem...

        The problem was with the large constant(learning rate) I used
        in the gradient descent algorithm. I was using sth. like 0.3-0.5,
        which basicly caused the weights and consequently outputs to blow up.

        I used very low learning rates like 0.001-0.005 and increased the
        number of nodes in the two hidden layers (10,8), and got better
        results but now convergence is extremely slow. In any case I think
        it's better than having outputs that blow up.

        But I am still curious if sth. else can be done to speed up the
        convergence. Somebody suggested the use of BAMs but I didn't
        quite understand the procedure.

        Regards,

        Kemal Ciliz
        mkciliz@cmx.npac.syr.edu
        olky@vax5.cit.cornell.edu