Xref: utzoo comp.ai.neural-nets:314 comp.ai:2591
Path: utzoo!attcan!uunet!seismo!sundc!pitstop!sun!decwrl!labrea!agate!helios.ee.lbl.gov!lll-tis!afit-ab!efrethei
From: efrethei@afit-ab.arpa (Erik J. Fretheim)
Newsgroups: comp.ai.neural-nets,comp.ai
Subject: Re: Learning arbitrary transfer functions
Summary: don't count on backprop
Message-ID: <718@afit-ab.arpa>
Date: 15 Nov 88 14:14:08 GMT
References: <399@uvaee.ee.virginia.EDU> <378@itivax.UUCP>
Reply-To: efrethei@blackbird.afit.af.mil (Erik J. Fretheim)
Distribution: na
Organization: Air Force Institute of Technology; WPAFB, OH
Lines: 38

In article <378@itivax.UUCP> dhw@itivax.UUCP (David H. West) writes:
In article <399@uvaee.ee.virginia.EDU> aam9n@uvaee.ee.virginia.EDU (Ali Mina
>
>I am looking for any references that might deal with the following
>problem:
>
>y = f(x);         f(x) is nonlinear in x
>
>Training Data = {(x1, y1), (x2, y2), ...... , (xn, yn)}
>
>Can the network now produce ym given xm, even if it has never seen the
>pair before?
>
>That is, given a set of input/output pairs for a nonlinear function, can a
>multi-layer neural network be trained to induce the transfer function
>                                                 ^^^
I don't know about non-linear functions but, I did try to train a net 
(back prop) to learn to compute sine(X) given X.  I trained it for two
weeks straight (virtually sole user) on an ELXSI.  The result was that in
carrying the solution to 5 significant decimal places I got a correct
solution 40% of the time.  Although this is somewhat better than random
chance, it is not good enough to be useful.  I will also note that the 
solution did not improve dramatically in the last week of training so I 
feel I can safely assume that the error rate would not decrease.  I also
tryied the same problem using a two's complement input/output and was able
to get about the same results in about the same amount of training.  The
binary representation needed a few more nodes though.  I was not able to
spot any significant or meaningful patterns in the errors the net was 
making and do not feel that reducing the number of significant decimal places
would help (even if it were meaningful) as the errors made were not 
consistantly in the last couple of digits, but rather were spread throughout
the number (in both binary and decimal representations).
Based on these observations, I don't think 
a net can be expected to produce any meaningful function.  Sure it can
do 1 + 1 and other simple things, but trips when it hits something not
easily exhaustively (or nearly exhaustively) trained. 

Just my opinion, but ...