Path: utzoo!utgpu!news-server.csri.toronto.edu!mailrus!uunet!ogicse!zephyr.ens.tek.com!tektronix!nosun!ohsuhcx!spackman
From: spackman@ohsuhcx.ohsu.edu (Dr. Kent Spackman)
Newsgroups: comp.ai.neural-nets
Subject: Maximum likelihood estimators for weights in feed-forward nets
Message-ID: <437@ohsuhcx.ohsu.edu>
Date: 10 Apr 90 19:54:22 GMT
Reply-To: spackman@ohsu.edu (Kent A. Spackman)
Distribution: usa
Organization: Oregon Health Sciences University, Portland
Lines: 26


I'm looking for references to articles that have anything to do with
the relationship between back-propagation and maximum likelihood
estimation, such as is done by logistic regression.

I have programmed a maximum likelihood estimator that trains a nnet by 
sort-of back-propagating likelihoods, resulting in a "multi-layer" logistic
regression.  It works well for simple problems (X-OR, etc.) involving
multiple layers and reproduces the results of single-layer logistic regression.

Has anyone done this before? Do you know of any references? 

Some background information:
Logistic regression is a commonly used statistical method in medicine.
It can be described as a method for maximum likelihood estimation
of the weights of a single-layer feed-forward neural network.  It uses
the logistic transfer function just like "ordinary" back-propagation.
Back-propagation does not do maximum likelihood estimation. Minimizing
the sum of squared errors is not the same as maximum likelihood when
using the logistic transfer function.

Reply to the net or directly via email to me:  spackman@ohsu.edu
Kent A. Spackman
Biomedical Information Communication Center
Oregon Health Sciences University
Portland, OR 97201