Path: utzoo!attcan!uunet!cs.utexas.edu!sdd.hp.com!ucsd!sdcc6!odin!demers
From: demers@odin.ucsd.edu (David E Demers)
Newsgroups: comp.ai.neural-nets
Subject: Re: Learning parity function by backprob.
Message-ID: <13122@sdcc6.ucsd.edu>
Date: 9 Oct 90 18:36:13 GMT
References: <4803@tuminfo1.lan.informatik.tu-muenchen.dbp.de>
Sender: news@sdcc6.ucsd.edu
Organization: CSE Dept., UC San Diego
Lines: 19
Nntp-Posting-Host: odin.ucsd.edu

In article <4803@tuminfo1.lan.informatik.tu-muenchen.dbp.de> li@kiss.informatik.tu-muenchen.de.informatik.tu-muenchen.dbp.de () writes:
>Parity functions may be realized by NN with one hidden layer (a simple
>solution was given in PDP-1). It is however a hard  problem to get such
>solution by back-propagation algorithm. I was able to train a  NN with
>backprog and some heuristcs to realize the P_4 (i.e. the parity
>function of 4 bits vectors, P_2 is the XOR function). The P_5 seems, by
>my experience, already to be too difficult to be learned by backprop,
>no matter how many layers and neurons are used.  Does someone know
>better results?

I am recalling Tim Ash's paper on his Dynamic Node Creation.
He used it on several problems, including parity 4 and 5, I
believe.  It is in a recent issue of Connection Science journal.

His method is essentially backprop, but with addition of nodes
(or layers!) when convergence either to the training set or a
test set stops.

Dave