Xref: utzoo comp.ai.neural-nets:303 comp.ai:2559
Path: utzoo!utgpu!attcan!uunet!ncrlnk!ncrcae!hubcap!gatech!uflorida!haven!uvaarpa!uvaee!aam9n
From: aam9n@uvaee.ee.virginia.EDU (Ali Minai)
Newsgroups: comp.ai.neural-nets,comp.ai
Subject: Learning arbitrary transfer functions
Message-ID: <399@uvaee.ee.virginia.EDU>
Date: 10 Nov 88 18:54:52 GMT
Organization: EE Dept, U of Virginia, Charlottesville
Lines: 44


I am looking for any references that might deal with the following
problem:

y = f(x);         f(x) is nonlinear in x

Training Data = {(x1, y1), (x2, y2), ...... , (xn, yn)}

Can the network now produce ym given xm, even if it has never seen the
pair before?

That is, given a set of input/output pairs for a nonlinear function, can a
multi-layer neural network be trained to induce the transfer function
by being shown the data? What are the network requirements? What
are the limitations, if any? Are there theoretical bounds on
the order, degree or complexity of learnable functions for networks
of a given type?

Note that I am speaking here of *continuous* functions, not discrete-valued
ones, so there is no immediate congruence with classification. Any attempt
to "discretize" or "digitize" the function leads to problems because the
resolution then becomes a factor, leading to misclassification unless
the discretizing scheme was chosen initially with careful knowledge
of the functions characteristics, which defeats the whole purpose. It
seems to me that in order to induce the function correctly, the network
must be shown real values, rather than some binary-coded version (e.g.
in terms of basis vectors). Also, given that neurons have a logistic
transfer function, is there a theoretical limit on what kinds of functions
*can* be induced by collections of such neurons?

All references, pointers, comments, advice, admonitions are welcome.
Thanks in advance,

                    Ali


Ali Minai
Dept. of Electrical Engg.
Thornton Hall
University of Virginia
Charlottesville, VA 22901

aam9n@uvaee.ee.Virginia.EDU
aam9n@maxwell.acc.Virginia.EDU