Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!maverick.ksu.ksu.edu!unmvax!ariel.unm.edu!hooey.unm.edu!bill
From: bill@hooey.unm.edu (william horne)
Newsgroups: comp.ai.neural-nets
Subject: Re: Approximate Realisation of Piecewise Linear Functions
Keywords: Approximation
Message-ID: <1990Aug7.213835.22024@ariel.unm.edu>
Date: 7 Aug 90 21:38:35 GMT
References: <1938@cybaswan.UUCP>
Sender: usenet@ariel.unm.edu (USENET News System)
Organization: University of New Mexico, Albuquerque
Lines: 32

In article <1938@cybaswan.UUCP> eeoglesb@cybaswan.UUCP (j.oglesby eleceng pgrad) writes:
>------------------------------------------------------------------------------
>
>1 Layer (no hidden nodes) -    gives a hyperplane that divides the input space
>                               in to two parts.
>3 Layer (two hidden layers) -  gives ANY piecewise linear division of the input
>                               space.
>OK thats the easy part, now
>
>2 Layer (one hidden layer)  -  ANY single piecewise linear convex region, 
>
>Now I can make some DISCONNECTED CONVEX regions and some DISCONNECTED 
>CONCAVE regions , however I don't think I can make all disconnected concave
>types of decision region with only one hidden layer.
>
>Can anybody rationalise the decision boundaries for one hidden layer nets
>with hardlimiting activation functions ? 
>

I had thought about this question a lot myself.  I realized what Lippmann
says wasn't right because I figured out how to make a donut shaped thing
with only a single hidden layer, but I couldn't figure out how to
generalize it.  I found some answers in,

J. Makhoul, A. El-Jaroudi, and R. Schwartz, "Formation of Disconnected
Decision Regions with a Single Hidden Layer", in IJCNN89, Vol I,
pp. 445-460.

I forget the details, but I thought it was a good paper at the time.
Hope this helps...

-Bill