Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!ucsd!pacbell.com!decwrl!shelby!csli!chrisley
From: chrisley@csli.Stanford.EDU (Ron Chrisley)
Newsgroups: comp.ai.neural-nets
Subject: Re: NN solution of non-deterministic problems. Doable or stupid?
Message-ID: <14755@csli.Stanford.EDU>
Date: 1 Aug 90 18:30:51 GMT
References: <14121@shlump.nac.dec.com>
Sender: chrisley@csli.Stanford.EDU (Ron Chrisley)
Organization: Center for the Study of Language and Information, Stanford U.
Lines: 36

Many people in the neural net/PDP community have ignored the non-deterministic
case of pattern-recognition.  I've seen talks/papers that try to provide
all-encompassing frameworks for pattern-recognition in nnets, and yet they
assume things like "there is a 0-error weight-state".  Of course, in truly
non-deterministic problems, there is no such thing as a state that never makes
mistakes.  All one can do is maximize the likelihood of correct
classification.  The same goes for prediction.

Yes, there are probably nets that predict the mean.  Nearest neighbor
classifiers will probably pick the mode (the output that was most frequently
associated with the input)

For example, if outputs are not predicted inches of rainfall, which is a 
continuous variable, but are instead small in number and discrete, such as
weather types like cloudy, windy, clear, etc., then one could use
a nearest-neighbor style classifier which would categorize an input to the
weather class that is most likely, given the history of inputs.

If you are interested in this latter type of discrete prediction, then I
suggest looking at Kohonen's work on LVQ (ICNN '88) as an introduction.

For the continuous case, you have to decide what kind of interpolation function
makes sense, it appears.  But I don't know much about this case.  Anyone else?

If using nnets for non-deterministic problems is "stupid" then nnets will be of
limited interest in many domains, such as speech.

Hope this helps.



-- 
Ron Chrisley    chrisley@csli.stanford.edu
Xerox PARC SSL                               New College
Palo Alto, CA 94304                          Oxford OX1 3BN, UK
(415) 494-4728                               (865) 793-484