Xref: utzoo comp.ai.neural-nets:942 alt.cyb-sys:24
Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!uwm.edu!gem.mps.ohio-state.edu!ginosko!aplcen!haven!uvaarpa!uvaee!aam9n
From: aam9n@uvaee.ee.virginia.EDU (Ali Minai)
Newsgroups: comp.ai.neural-nets,alt.cyb-sys
Subject: Generalization Criteria
Message-ID: <506@uvaee.ee.virginia.EDU>
Date: 18 Sep 89 20:39:58 GMT
Organization: EE Dept, U of Virginia, Charlottesville
Lines: 32



Following the very interesting but rather inconclusive discussion of
generalization on comp.ai.neural-nets, I have a question. 

Given a *finite* set of input-output values, an estimator/approximator
is used to induce a "reasonable" model for the system that generated
the data. While there are many measures of "goodness of fit" with
respect to the given data set (e.g. mean squared error), there seems
to be no universally accepted measures for the "generalization" achieved.
So far, I have just seen variants of two approaches:

1) Using a "test set" of data not seen by the estimator during training,
   and comparing the error on this set with that on the training set.

2) Looking at the statistical properties of the estimator's responses on
   training and testing data to see how similar they are.

I would appreciate any references that describe/use/propound any other
criteria for generalization. In particular, I am interested in the
philosophical issues behind this problem and its implications for
the scientific method.

Thank you.

Ali Minai
Dept. of Electrical Engg.
Thornton Hall
University of Virginia
Charlottesville, VA 22903

aam9n@uvaee.ee.virginia.edu