Path: utzoo!utgpu!jarvis.csri.toronto.edu!cs.utexas.edu!asuvax!hrc!gtx!al
From: al@gtx.com (Alan Filipski)
Newsgroups: comp.ai.neural-nets
Subject: updating neural nets
Message-ID: <1153@gtx.com>
Date: 7 Dec 89 20:34:13 GMT
Reply-To: al@gtx.UUCP (Alan Filipski)
Organization: GTX Corporation, Phoenix
Lines: 23

Suppose a net has been trained (e.g., using backprop) on several
thousand samples and a few more samples come along.  One could retrain
the net using all (several thousand + few) samples, or one could
somehow modify the weights using only the few new samples.The risk
of the latter approach is that the gain in learning about the
few new samples may be outweighed by degraded performance on the old
samples.  The advantage, of course, is that all the old samples do not
have to be retained.

To look at a simple analog, suppose we have an estimate of the mean of
1000 numbers, and then 3 more numbers come in.  it is easy to compute
the mean of all 1003 numbers withous seeing the first 1000 again--
their mean, along with the count of how many there were, provides a
sufficient statistic for updating the mean.

Does anyone know of any work on updating neural nets to account for new
samples without completely retraining them?


  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 ( Alan Filipski, GTX Corp, 8836 N. 23rd Avenue, Phoenix, Arizona 85021, USA )
 ( {decvax,hplabs,uunet!amdahl,nsc}!sun!sunburn!gtx!al         (602)870-1696 )
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~