Path: utzoo!attcan!uunet!cs.utexas.edu!news-server.csri.toronto.edu!neuron.ai.toronto.edu!ai.toronto.edu!tap
From: tap@ai.toronto.edu (Tony Plate)
Newsgroups: comp.ai.neural-nets
Subject: Re: Using deltas at the input layer
Message-ID: <90Jul26.123404edt.304@neuron.ai.toronto.edu>
Date: 26 Jul 90 16:34:24 GMT
References: <12039@sdcc6.ucsd.edu>
Organization: Department of Computer Science, University of Toronto
Lines: 30

In article <12039@sdcc6.ucsd.edu> demers@beowulf.ucsd.edu (David Demers) writes:
>I'm curious to know if anyone has done anything
>with computation of deltas for input units.  I'm
>aware of the paper by Risto Mikkulainen & Mike Dyer
>in 1988 Connectionist Models Summer School proceedings.
>
>Anyone else try anything using a normal backpropogation
>delta at the input layer?
>
>Dave DeMers

All networks which use a positional (1 on in n) encoding of input
can be considered as doing what you are talking about.

Just think of the second or hidden layer as being the input,
and the weights from one of the positionally encoded units
as the encoding for that item.

For an earlier example, see Hinton's family tree paper, in proc.
of the 8'th Cog Sci conf, 1986.

You could think of NETtalk as a version of this that takes context
into account (i.e., representing the central letter in its context).

Look for positionally encoded inputs and outputs to identify
more examples.  It gets more interesting at the output where
the distributed representation sometimes has to encode the
possibility that are several possible answers.

Tony Plate