Newsgroups: comp.ai.neural-nets
Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!omlinc
From: omlinc@cs.rpi.edu (Christian Omlin)
Subject: Fault-Tolerance of NN
Message-ID: <lx6gjll@rpi.edu>
Keywords: fault-tolerance, fault-recovery, fault-detection
Sender: omlinc@cs.rpi.edu
Nntp-Posting-Host: cs.rpi.edu
Organization: Rensselaer Computer Science, Troy NY
Date: 6 May 91 16:25:03 GMT
Lines: 48

Hi !

A few papers have appeared recently dealing with retraining (using
backpropagation) as a strategy by which feedforward NN's can recover
from faults such as neuron stuck-at faults. A few questions come to
my mind:
 
 1. Often, retraining a network is claimed to be easier (i.e. faster)
    than training the original, flawless network with small random initial
    weights. My experiments show that a network is not guaranteed to
    relearn the intended I/O mapping, i.e. it a network may get trapped
    in a local minimum. Is relearning inherently easier than learning
    assuming there are enough units in the hidden layer ?

2. Suppose we can retrain a network, we are not guaranteed that the
   network exhibits the same characteristics (e.g. generalization)
   which may have been one of the criteria during the design of the NN.
   Wouldn't it be more reasonable to detect structural damages of the
   NN before it is used in an application and repair the damage ? 
   (This would require some method for detecting such faults.)

3. Giving a NN a retraining capability, certainly requires
   additional hardware and information about the training set. How
   big is the additional cost of hardware of a NN with retraining
   capability as opposed to a non-retrainable NN ?

4. It seems fault-tolerance is not an inherent property of NN, rather
   they have to be designed with fault-tolerance in mind. There seem
   to be two possibilities for improving the fault-tolerant behavior:
   changes in the architecture and a changes in the training procedure.
   Which of the two is more effective ? 

Any comments are appreciated.

Christian

----------------------------------------------------------------------------
Christian W. Omlin			

office:                                 home:
Computer Science Department             Foxberry Farm
Amos Eaton 119                          Box 332, Route #3
Rensselaer Polytechnic Institute        Averill Park, NY 12018
Troy, NY 12180 USA                      (518) 766-5790
(518) 276-2930                        

e-mail: omlinc@turing.cs.rpi.edu
----------------------------------------------------------------------------