Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!uwm.edu!uakari.primate.wisc.edu!ctrsol!cica!iuvax!rutgers!dptg!att!cbnewsl!apr
From: apr@cbnewsl.ATT.COM (anthony.p.russo)
Newsgroups: comp.ai.neural-nets
Subject: Re: : Step Function. Biases are necessary
Summary: do we need a new definition of generalization?
Keywords: learning,generalization
Message-ID: <1866@cbnewsl.ATT.COM>
Date: 14 Sep 89 11:57:51 GMT
References: <1693@hydra.riacs.edu>
Organization: AT&T Bell Laboratories
Lines: 42

Doug Danforth wrote:
> 
>      The reason for this  preamble is to show that reading from (presenting
> an input pattern to) a sparse distributed memory (a neural net) can indeed
> produce output which is a "generalization".  The generalization can take
> the form of: (A) what's the most similar thing to this pattern that I have
> seen before, or (B) what is the Platonic ideal of this fuzzy pattern?
> 
***

I don't really want to belabor this discussion much longer, but I don't
see any generalization in your example. What has been generalized? On the
surface, you're saying the fuzzy pattern has been generalized to the
"ideal" of the stored templates, but I think that the memory has simply
calculated a predetermined distance function. In light of the ongoing
discussion, there are two ways to view this:

1) the energy function is the "bias" we've been talking about. But in this case
there is no other learning going on, so the generalization is due solely
to the bias. If you accept this, then every classifier generalizes, and
we need a better definition of generalization.

2) The energy function is not this "bias;" the biases are instead due
to the topology etc. of the network. In this case, the rule for similarity
was "given" to the network, not extracted by it.

It would be nice and clean if the second case were correct, but I 
wouldn't bet on it.

> By "rule extraction" I assume you mean in analogy to human concious throught
> where one can articulate the "rule" that one has discovered.  This is an ongoing
> area of debate.  Is it necessary to "interpret" the connection weights or just
> evaluate the performance of the system?   IMHO, that depends upon your goals.
                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You're right. 

 ~ tony ~

	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
	~  	 Tony Russo		" Surrender to the void."	~
	~   apr@cbnewsl.ATT.COM						~
	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~