Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!csd4.csd.uwm.edu!gem.mps.ohio-state.edu!ginosko!usc!apple!sun-barr!rutgers!dptg!att!cbnewsl!apr
From: apr@cbnewsl.ATT.COM (anthony.p.russo)
Newsgroups: comp.ai.neural-nets
Subject: Re: : Step Function
Summary: should we teach biases or give examples?
Keywords: learning,generalization
Message-ID: <1750@cbnewsl.ATT.COM>
Date: 5 Sep 89 11:26:00 GMT
References: <1060@rex.cs.tulane.edu> <6980@sdcsvax.UCSD.Edu> <11308@boulder.Colorado.EDU>
Organization: AT&T Bell Laboratories
Lines: 30

In article <11308@boulder.Colorado.EDU>, bill@boulder.Colorado.EDU writes:
> 
>   Is it necessary to have a bias in order to be able to learn?
> 
>   Not always. [...]
>   
>   If, though, there are inputs not included in the training set,
> then the device can only produce responses by somehow "generalizing"
> from what it has seen, and the rules it uses for generalization
> constitute a bias.  (I actually prefer the word "heuristic" to
> "bias".  What I am saying here is that any device that learns from
> incomplete data must use some sort of heuristic.)
> 
From what I have thought about and read here, it seems that learning
actually constitutes learning *biases* and nothing else (unless it is
memorization).

*IF* this is true (and I'm not sure it is), then for a classifier,
from an information-theoretic point of view, it then makes the most sense
to give the network examples that are on the borderline
of a rule or class. These "borderline" patterns should contain more
information about biases than good examples of a class.

In ways this makes sense -- it is akin to telling the network only where the
boundaries between classes are.

	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
	~  	 Tony Russo		" Surrender to the void."	~
	~   apr@cbnewsl.ATT.COM		   put disclaimer here		~
	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~