Path: utzoo!yunexus!ists!jarvis.csri.toronto.edu!mailrus!ncar!boulder!mathis
From: mathis@boulder.Colorado.EDU (Don Mathis)
Newsgroups: comp.ai.neural-nets
Subject: Re: Neural Network for ranking football teams.
Summary: Rebuttal to criticism
Message-ID: <14017@boulder.Colorado.EDU>
Date: 17 Nov 89 19:43:00 GMT
Article-I.D.: boulder.14017
References: <13645@boulder.Colorado.EDU> <80663@linus.UUCP>
Sender: news@boulder.Colorado.EDU
Reply-To: mathis@boulder.Colorado.EDU (Don Mathis)
Organization: University of Colorado, Boulder
Lines: 68


    In article <80663@linus.UUCP> sdo@faron.UUCP (Sean D. O'Neil) wrote a
criticism of the football net, in which was said:
>>For those of you who know a bit about neural nets, this is a constraint 
                                                                     ^^^^^^^^^^
>>satisfaction network that settles on what I call a "quality  value" for each 
 ^^^^^^^^^^^^ ^^^^^^^                                             

>Sometimes known as a Hopfield network.

No, it's not a Hopfield net.  Hopfield nets use binary threshold units and 
asynchronous updates.  This net has continuous-valued activations and 
updates the units in parallel.  It's doing something closer to "pure" gradient-
following.

>My problem is this:  there is no, repeat no, reason to use a Hopfield network
>to solve this problem.  

How about for fun?   Your statement is like saying "There is no reason to use 
Pascal to solve this problem".  It's just a way to implement an algorithm.

>					It is an *unconstrained* minimization of a
>quadratic function.  This is a trivial problem to solve using first-year
>calculus.  Take the derivative of the quadratic function, set it equal
>to zero, and solve the set of linear equations to get the quality values.
>This *exactly* satisfies Don's function minimization criterion.

You're right, in that the net is finding a least-squares solution to an 
overdetermined linear system.  

>Note that I am NOT saying that Hopfield or constraint satisfaction networks
>have no use.  Often one wishes to constrain values that the outputs of the
>network can take on.  This is usually done implicitly by shaping the transfer 
>or activation function in some way---typically a sigmoidal shape is used.
>In such cases, one CANNOT take the algebraic approach I described above and
>it is often the case that the easiest solution technique is to run the
>network and let it converge.  However, such is not the case here.

I agree.

> ...Thus, there is not a unique solution, but rather
>a whole space of solutions, all equally valid.  The neural network gives
>us a particular solution, but if we initialized it with some starting values
>other than 0.0, we would get a completely different solution.

Yes, there are an infinite number of least-squares solutions.  But there IS a 
unique least-squares solution of minimum L2-norm.  What I do is take the 
solution the network finds, and use it to obtain the solution of minimum L2-
norm.  This composite algorithm is independent of initial values.

>In conclusion, I think that it is important to look at what's really going
>on when we set up a neural network to solve a problem.  There seems to
>be an attitude that the neural network does something mystical and
>I want to get away from that.  In many cases, as in this one, the network
>is merely mimicking some straightforward mathematical procedure that
>can best be handled with standard techniques.

Well, I think you perceive this attitude because you work at Mitre.  Nothing 
against Mitre specifically, but I've found that in the BUSINESS of neural nets, 
those who can best lie to their prospective customers about the magic of 
neural nets make the most money, and they will continue to do so until the 
myths are exposed.     But this problem only exists in industry - where 
people do things without thinking first.  I would hope that the people who 
read this bboard do enough thinking to sort these things out for themselves. 
 The whole "magic" issue is YOUR problem - you struggle with it.  I'm not 
trying to snow anyone - I'm just trying to have fun.

-Don