Path: utzoo!attcan!uunet!mcsun!hp4nl!fwi.uva.nl!smagt
From: smagt@fwi.uva.nl (Patrick van der Smagt)
Newsgroups: comp.ai.neural-nets
Subject: Re: Reference for sufficiency of two layers
Message-ID: <1301@carol.fwi.uva.nl>
Date: 3 Oct 90 08:10:28 GMT
References: <12626564934017@osu-20.ircc.ohio-state.edu>
Sender: news@fwi.uva.nl
Reply-To: smagt@fwi.uva.nl (Patrick van der Smagt)
Organization: FWI, University of Amsterdam
Lines: 76

In article <12626564934017@osu-20.ircc.ohio-state.edu> WHITELEY-J@osu-20.ircc.ohio-state.edu (J. Whiteley) writes:
>I'm looking for a reference which shows that two layers are sufficient
>to approximate any function using a BP network.  All I've dug up so
>far is:
>
>Cybenko, G.  Approximation by superpositions of a sigmoidal function.
>Unpublished manuscript, Oct. 1988.
>
There are three articles proving this.  The first one is the most
complete:

	%A K. Hornik
	%A M. Stinchcombe
	%A H. White
	%T Multilayer feedforward networks are universal approximators
	%J Neural Networks
	%V 2
	%N 5
	%D 1989
	%P 359--366

	%A G. Cybenko
	%T Approximation by superpositions of a sigmoidal function
	%J Mathematics of Control, Signals, and Systems
	%V 2
	%N 4
	%D 1989
	%P 303--314

	%A K.-I. Funahashi
	%T On the approximate realization of continuous mappings
	by neural networks
	%J Neural Networks
	%V 2
	%N 3
	%D 1989
	%P 193--192

Furthermore, the next paper proves something similar but now
for hidden units with a Gaussian activation function (very, very useful
for function approximation; I experience faster learning and greater
accuracy, although be careful with initial weight values and
learning rates):

	%A E. J. Hartman
	%A J. D. Keeler
	%A J. M. Kowalski
	%T Layered Neural Networks with Gaussian Hidden Units
	as Universal Approximations
	%J Neural Computations
	%V 2
	%N 2
	%D Summer 1990
	%P 210-215

Send me an email if you want a proof for binary networks (from Perceptrons).


Patrick van der Smagt                                               /\/\
                                                                    \  /
Organization: Faculty of Mathematics & Computer Science             /  \
              University of Amsterdam, Kruislaan 409,            _  \/\/  _
              NL-1098 SJ  Amsterdam, The Netherlands            | |      | |
Phone:        +31 20  525 7466                                  | | /\/\ | |
Telex:        10262 hef nl                                      | | \  / | |
Fax:          +31 20  592 5155                                  | | /  \ | |
email:        smagt@fwi.uva.nl                                  | | \/\/ | |
                                                                | \______/ |
                                                                 \________/

								    /\/\
                                                                    \  /
                                                                    /  \
                                                                    \/\/
``The opinions expressed herein are the author's only and do
not necessarily reflect those of the University of Amsterdam.''