Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!clyde.concordia.ca!uunet!aplcen!samsung!shadooby!umich!itivax!dhw
From: dhw@itivax.iti.org (David H. West)
Newsgroups: comp.ai.neural-nets
Subject: Re: back-prop NNs and `SAS' regression!
Message-ID: <4673@itivax.iti.org>
Date: 19 Dec 89 17:33:15 GMT
References: <220700005@uxe.cso.uiuc.edu> <1989Dec18.210859.23621@wuche2.wustl.edu> <21539@uflorida.cis.ufl.EDU>
Reply-To: dhw@itivax.UUCP (David H. West)
Organization: Industrial Technology Institute
Lines: 30

In article <21539@uflorida.cis.ufl.EDU> fishwick@fish.cis.ufl.edu (Paul Fishwick) writes:
|In article <1989Dec18.210859.23621@wuche2.wustl.edu> joshi@wuche2.UUCP (Amol Joshi) writes:
|>	the techniques of multi-variable analysis are not suited to non-linear
|>	phenomena and many real problems are non-linear.
|>	even though non-linear regressions can treat non-linear phenomena, they
|>	require that the structure of the math model be prefixed.
|>	it is in these cases that backprop nets would be more useful.
|
|You say that in regression that the "structure of the model be prefixed"
|however I will debate this assumption -- the structure of a set of
|equations is no more prefixed than a neural network model. A neural
|network is a set of equations shown in a graphical syntactic form.
|It is just as easy to add and delete terms/equations as it is to add/delete
|nodes, etc. The equational equivalent of removing a link is to make
|zero a parameter.

... and to reduce the rank of the model by one.  Variable-rank
methods are essentially a recent development in statistics and
optimization [1960s and later - yes, that's "recent" :-(  ], and are
not yet part of the repertoire of many (perhaps most) practitioners
and software packages, even in the methods' simpler linear form.  
Nonlinear variable-rank methods are still a research problem.

One advantage of viewing neural-net training in the light of
statistics and optimization is to focus attention on the fact that
the standard sigmoidal transfer function is no less a mere
convention than is the Gaussian probability density, and equally a
choice to be made consciously rather than by default.

-David West       dhw@iti.org