Path: utzoo!utgpu!jarvis.csri.toronto.edu!rutgers!usc!orion.cf.uci.edu!uci-ics!ucla-cs!mara!mickey.cognet.ucla.edu!kennel From: kennel@mickey.cognet.ucla.edu (Matthew Kennel) Newsgroups: comp.ai.neural-nets Subject: Re: 3-Layer versus Multi-Layer Message-ID: <77@mara.cognet.ucla.edu> Date: 3 Jul 89 18:06:42 GMT References: <3417@cosmo.UUCP> Sender: news@mara.cognet.ucla.edu Reply-To: kennel@mickey.cognet.ucla.edu.UUCP (Matthew Kennel) Organization: none Lines: 40 In article merrill@bucasb.bu.edu (John Merrill) writes: >One reference to such a result is > >Funahashi, K. (1989). "On the Approximate Realization of Continuous >Mappings by Neural Networks", {\bf Neural Networks} (2) 183-192. > >There are actually several different theorems which prove the same >thing, but Funahashi's is the first that I know of which does it from >standard sigmoid semi-linear nodes. > >-- >John Merrill | ARPA: merrill@bucasb.bu.edu >Center for Adaptive Systems | >111 Cummington Street | >Boston, Mass. 02215 | Phone: (617) 353-5765 I recently saw a preprint by some EE professors at Princeton who made a constructive proof using something called the "inverse Radon transform", or something like that. What I think the subject needs is work on characterizing the "complexity" of continuous mappings, w.r.t. neural networks--- i.e. how many hidden units (free coefficients) are needed to reproduce some mapping with a certain accuracy? Obviously, this depends crucially on the functional basis and architecture of the network---we might be able to thus evaluate various network types on their power and efficiency in a practical way, and not just formal (i.e. given infinite hidden neurons). My undergrad thesis adviser, Eric Baum, has been working on this type of problem, but for binary-valued networks, i.e. networks that classify the input space into arbitrary categories. The theory is quite mathematical---as a "gut feeling" I suspect that for continous-valued networks, only approximate results would be possible. Matt Kennel kennel@cognet.ucla.edu