Xref: utzoo comp.ai.neural-nets:620 sci.philosophy.tech:1110
Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!rutgers!njin!princeton!phoenix!mbkennel
From: mbkennel@phoenix.Princeton.EDU (Matthew B. Kennel)
Newsgroups: comp.ai.neural-nets,sci.philosophy.tech
Subject: Re: request for philosophic reactions to connectionism
Keywords: connectionism philosophy materialism representations
Message-ID: <7894@phoenix.Princeton.EDU>
Date: 21 Apr 89 21:41:55 GMT
References: <370@eurtrx.UUCP> <18496@gatech.edu>
Reply-To: mbkennel@phoenix.Princeton.EDU (Matthew B. Kennel)
Followup-To: comp.ai.neural-nets
Organization: Princeton University, NJ
Lines: 52

In article <18496@gatech.edu> myke@gatech.UUCP (Myke Rynolds) writes:
>
>I think that BAMs (bi-direction associative memories) and it's conceptual
>parent, ART (adaptive resonance theory) give a profound critique of the
>connectionist models. Grossberg, the inventer of ART way back in '76, goes
>into great detail about how nothing anyone in the connectist school of thought
>has said is new, or even as powerful as what already exists! ART is proven
>to converge on any complexity of input, no connectionist model can claim this.
>They can learn only by limiting the complexity of the input, thus the failure
>of bp to deal with large and complex systems.
>For all its greater power, it is much much simpliar than these other models
             ^^^^^^^^^^^^^ 
>that cloud the issue with ad hoc hockus pockus. Grossberg's model is nothing
>more than matrix multiplication. You take a vector forward through a weight
           ^^^^^^^^^^^^^^^^^^^^^^  
>matrix, then take it backwards through it. When it resonates on the correct
>answer you're done. The most obvious way to get a weight matrix to satisfy
>this problem on a series of such vectors is to stack them in a matrix and
>do linear algebra. Walla!
    ^^^^^^ 
    Voila!

That's exactly the point.  For linear problems, than I have no doubt that
classical algorithms (linear systems of equations) should work better than
gradient descent (BP), with the whole shebang of nice rigorous results, but
the whole point is that back-prop tries to learn general non-linear
transformations that AREN'T matrix multiplications.
For some kinds of associative memory something like ART
may be fine, but associative memory isn't the whole story.  It's
generalization (i.e. high-dimensional interpolation) which is the the most
interesting aspect of multi-layer perceptrons.

Can something like a BAM network be more efficient than an "encoder"
type of perceptron in terms of the number of connections?  

>An article on BAMs can be found in a Byte from last year.
>BTW, Grossberg has three Ph.D's, two of which are in math and neurophysiology.
>Connectionists are generally psychologists and computer scientists who do not
>appreciate the deeper simplicity of math under the outer tremendous diversity.

I've never been able to discern the deeper simplicity of math in any ART paper
that I've seen (which is very few, I must admit); back-prop is 

>-- 
>Myke Rynolds
>School of Information & Computer Science, Georgia Tech, Atlanta GA 30332
>uucp:	...!{decvax,hplabs,ncar,purdue,rutgers}!gatech!myke
>Internet:	myke@gatech.edu


Matt Kennel
mbkennel@phoenix.princeton.edu