Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!uunet!jarthur!nntp-server.caltech.edu!tylerh
From: tylerh@nntp-server.caltech.edu (Tyler R. Holcomb)
Newsgroups: comp.ai.neural-nets
Subject: Re: Several hidden layers in feed-forward networks
Message-ID: <1991Jan8.205243.17280@nntp-server.caltech.edu>
Date: 8 Jan 91 20:52:43 GMT
References: <7165.27885d62@abo.fi> <1991Jan7.202202.12266@murdoch.acc.Virginia.EDU> <1991Jan8.091631.16219@warwick.ac.uk>
Organization: California Institute of Technology, Pasadena
Lines: 32

esrmm@warwick.ac.uk (Denis Anthony) writes:

>In article <1991Jan7.202202.12266@murdoch.acc.Virginia.EDU> aam9n@helga0.acc.Virginia.EDU (Ali Ahmad Minai) writes:
>>In article <7165.27885d62@abo.fi> vt_ai@abo.fi writes:
>>

...

>>an output layer neuron in a single hidden-layer net requires about 2n
>>hidden neurons to compose a function with n modes (peaks).

>Why 2n ? Is this emprical, or based on maths ? Or is it obvious,
>ie. 2n to form n peaks and n troughs. Apologies if I am being
>a bit dim.

>Denis

The 2n rule actually has a very strong theoretical basis. If one
views each hidden layer as performing a topological transformation
on a n-sphere (an odd, but compeletely equivalent view
of feed forward neural computation), then one can justify the 2n
rule from  Whitney's
Theorem of differential geometry.  This was demonstrated
in an unpublished work of John Cortese (it was a term project
for one of his classes).

To get more infomation, send e-mail to jcort@tybalt.caltech.edu.

I have a copy of the paper, but I do not have the right to
be distributing an unpublished work that isn't mine.

happy theorizing!