Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!ncar!gatech!myke
From: myke@gatech.edu (Myke Rynolds)
Newsgroups: comp.ai.neural-nets
Subject: Re: request for philosophic reactions to connectionism
Keywords: connectionism philosophy materialism representations
Message-ID: <18504@gatech.edu>
Date: 22 Apr 89 01:12:26 GMT
References: <370@eurtrx.UUCP| <18496@gatech.edu> <7894@phoenix.Princeton.EDU>
Reply-To: myke@gatech.UUCP (Myke Rynolds)
Organization: School of Information and Computer Science, Georgia Tech, Atlanta
Lines: 33

Matthew B. Kennel writes:
|That's exactly the point.  For linear problems, than I have no doubt that
|classical algorithms (linear systems of equations) should work better than
|gradient descent (BP), with the whole shebang of nice rigorous results, but
|the whole point is that back-prop tries to learn general non-linear
|transformations that AREN'T matrix multiplications.
|For some kinds of associative memory something like ART
|may be fine, but associative memory isn't the whole story.  It's
|generalization (i.e. high-dimensional interpolation) which is the the most
|interesting aspect of multi-layer perceptrons.

Ah! It is if you are dealing with real valued neurons, which BP gives the
fascade of doing, but infact it only uses the high and low end of the range
and is thus binary. With binary neurons, non-linear models are not one iota
more powerful. Infact, they only increase the complexity of the alg. I'm
working on a sparse linear equation solver with binary compaction to function
as a BAM type associative memory. I want to beat the masters at chess.
(ALL my facualty are excited by my ideas! Thats to say, I'm not a crank)

|Can something like a BAM network be more efficient than an "encoder"
|type of perceptron in terms of the number of connections?  
Its an associative memory, not an encoder. Night and day.

|>Connectionists are generally psychologists and computer scientists who do not
|>appreciate the deeper simplicity of math under the outer tremendous diversity.
|I've never been able to discern the deeper simplicity of math in any ART paper
|that I've seen (which is very few, I must admit); back-prop is 
Thats because the dude is clueless about people.
-- 
Myke Rynolds
School of Information & Computer Science, Georgia Tech, Atlanta GA 30332
uucp:	...!{decvax,hplabs,ncar,purdue,rutgers}!gatech!myke
Internet:	myke@gatech.edu