Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!mit-eddie!wuarchive!zaphod.mps.ohio-state.edu!know!news.cs.indiana.edu!ux1.cso.uiuc.edu!bradley2.bradley.edu!pwh
From: pwh@bradley2.bradley.edu (Pete Hartman)
Newsgroups: comp.ai.neural-nets
Subject: visualization of Weight "meanings"
Keywords: partitioning vector space
Message-ID: <1990Dec14.201607.5832@bradley2.bradley.edu>
Date: 14 Dec 90 20:16:07 GMT
Distribution: usa
Organization: Bradley University
Lines: 41

I hope this doesn't just serve to demonstrate my ignorance of the
field, but....

I am currently working my way through the Rummelhart and McClelland
PDP books, as sort of perparatory background to verify for myself
that I have the interest and ability to go on to grad school to study
Neural Nets.  I was reading the math chapters, basically refreshers
on linear algebra, vectors and matrices, and ran across some interesting
(and new to me) concepts.

The author describes the activation of a unit in terms of the
dot product of the input vector and the weight vector.  And a set 
of units in terms of a weight matrix made up of the various vectors.  
This I've seen around.  However, it was pointed out that the activation 
is actually (using a geometric interpretation of the dot product) a 
measure of how close the input vector matches the weight vector.  
Conceptually, the unit can be perceived as partitioning the input 
space into inputs that provide positive (or zero) activations, and 
inputs that proved negative activations.  (and I suppose you could 
see it in terms of more gradations than strict partitioning, too)

This is probably old news, but I was wondering....has anyone done
any work at representing such partitionings graphically?  For example,
in a very simple space where inputs were only 3 dimensional, a set
of units could be envisioned as partitioning the volume into seperate
regions.  I would think that these regions could provide insight to
the "meanings" of the weights.  Perhaps even after going through a 
training process they could be used to analyze the "final" states
to see exactly what was going on.  I suppose the hardest part would
be finding a way of graphically representing regions of dimensionality
greater than 3, (from what I've seen, the vast majority of problems
are of higher dimension), since the partitioning seems fairly simple
to find given enough crunching.

If such work has been done, could someone point me to it?  If not,
does it seem worthy of thinking about, or is this just idle whimsy
of someone not yet aware enough of the problems to see how useless
the idea is?
-- 
-----
Pete Hartman		pwh@bradley.bradley.edu			Haazavaa?