Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!purdue!ames!oliveb!apple!voder!berlioz!andrew
From: andrew@berlioz (Lord Snooty @ The Giant Poisoned Electric Head)
Newsgroups: comp.ai.neural-nets
Subject: Re: Biological Reality and Backpropagation
Keywords: Backpropagation, Biology, NETtalk
Message-ID: <190@bach.nsc.com>
Date: 9 May 89 01:41:40 GMT
References: <2518@bucsb.UUCP> <180@bach.nsc.com> <7487@spool.cs.wisc.edu>
Organization: National Semiconductor, Santa Clara
Lines: 62

In article <7487@spool.cs.wisc.edu>, neves@ai.cs.wisc.edu (David M. Neves) writes:
> In article <180@bach.nsc.com> andrew@berlioz (Lord Snooty @ The Giant Poisoned Electric Head) writes:
> >On the contrary, Stephen Grossberg ("Neural Networks and Natural
> >Intelligence") takes the opposed view - that weight transport, required
> >for BP, is a physically implausible mechanism.
> What weight transport is required by back propagation?  I really see
> no reason to transfer weights anywhere.  If he is talking about
> computation and storage and movement within the neuron I didn't
> realize we were so knowledgeable about neurons that we could already
> rule out large classes of models.

Well David, I will quote without permission from the tome in that case.
From section 17 in the chapter "Competitive Learning", p235 et seq.:

   "Comparing Adaptive Resonance and Back Propagation Models
   [..an enumeration of characteristics favourable to ART, unfavourable to BP..]
   C. Weight Transport or Top-Down Template Learning
   In both a BP model and an ART model, both bottom-up and top-down LTM traces
   exist. In a BP model (see Fig) the top-down traces in F4->F5 pathways are
   formal transports of the learned F2->F3 traces. In an ART model (see Fig)
   the top-down traces in F2->F1 pathways are directly learned by a realtime
   associative process. These top-down LTM weights are not transports of the
   learned LTM traces in the F1->F2 pathways, and they need not equal these
   bottom-up LTM traces. Thus an ART model is designed so that both bottom-up
   and top-down learning are part of a single information processing 
   hierarchy, which can be realised by a locally computable realtime process."

The ART Fig. is well-known, and the BP Fig. is as follows:

expected outputs	         --------------------  +
       | +         --------------| differentiator F6 |<------------
       \/          |             ---------------------        ----|-------
----------------- +|                                          | actual    |
| error signals |<-|                                          | outputs F3|
|	F4      |<--------------------------------------------|------------
--------------|-| -			learning signal		   /\
	|     |-------------------------------------------------->>|
	|		(weight transport)			   |
	|<---------------------------------------------------------|
	|							   |
	|			----------------- +		-----------
	|		--------| differentiator|<--------------| hidden   |
	|		|	|	F7	|		| units    |
	\/		|	-----------------		|    F2    |
 --------------- +	| 					------------
| error signals |<------      learning signal			   /\
|    F5		|------------------------------------------------>>|
-----------------						   |
								----------
								| inputs |
								|   F1   |
								----------
"Circuit diagram of BP model: in addition to the processing levels F1, F2,
F3, there are also levels F4, F5, F6 and F7 to carry out the computations
which control the learning process. The transport of learned weights from
the F2->F3 pathways to the F4->F5 pathways shows that this algorithm
cannot represent a learning process in the brain".
-- 
Andrew Palfreyman 		USENET: ...{this biomass}!nsc!logic!andrew
National Semiconductor M/S D3969, 2900 Semiconductor Dr., PO Box 58090,
Santa Clara, CA 95052-8090 ; 408-721-4788 		there's many a slip
							'twixt cup and lip