Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10.3 4.3bsd-beta 6/6/85; site sdics.UUCP
Path: utzoo!watmath!clyde!burl!ulysses!allegra!mit-eddie!genrad!decvax!ittatc!dcdwest!sdcsvax!sdics!munro
From: munro@sdics.UUCP (Paul Munro)
Newsgroups: net.cog-eng
Subject: Re: An idea...
Message-ID: <159@sdics.UUCP>
Date: Mon, 17-Mar-86 02:06:58 EST
Article-I.D.: sdics.159
Posted: Mon Mar 17 02:06:58 1986
Date-Received: Wed, 19-Mar-86 04:03:46 EST
References: <2008@hao.UUCP>
Reply-To: munro@sdics.UUCP (Paul Munro)
Followup-To: net.cog-eng
Distribution: net
Organization: U.C. San Diego, Institute for Cognitive Science
Lines: 77
Keywords: parallel distributed processing (PDP) / connectionist nets
Summary: PDP network mappings

In article <2008@hao.UUCP> bill@hao.UUCP (Bill Roberts) writes:
>
>	Suppose you have a function defined as follows:
>	   f:MS * context ------> Behavior
>
>	where 
>	 *  ::= a projection operation that takes a subset of MS and 
>		"projects" this subset onto the space defined by context.
>
>	 MS ::= a compostion of abstract data types,
>
>         context ::= an enviroment in which the ADTs are "applied",
>
>	 Behavior::= a class of "observable responses".
>

The PDP (parallel distributed processing) or connectionist approach
to cognitive science is an instance of the more general framework proposed
above.  PDP networks are essentially nonlinear mappings of input arrays
("context") to output arrays ("behavior") and hence fulfill the role of the
MS (mental state?) in the proposed scheme.  Many, if not most, PDP models
incorporate rules governing the dynamics of network parameters and hence
exhibit various kinds of learning phenomena.   In general, such learning
mechanisms tend to optimize some quantity with respect to the statistics of
the input environment.  Indeed they are often designed with some optimization
criterion built in (such as to minimize the error between the observed
and desired output characteristics).

In the language of the original posting, PDP learning theories generally
contain two functions one for processing (F) and one for learning (L):

 	   F: MS * context ------> Behavior

	   L: [old MS, context, behavior] ------> new MS



Here is a bibliography of some books and articles you may 
find interesting.  You may well already be familiar with some or all of
these.  First I should mention two journals that include articles concerned
with neural network processes: Biological Cybernetics (Springer) and the
IEEE Proceedings in System, Man, & Cybernetics.  The September 1983 issue
of this journal is particularly rich is network brain models.

Some books you might find interesting are:

T. Kohonen:   Self-Organization ans Associative Memory.  Springer (1984)
G. Palm:      Neural Assemblies.  Springer (1982)

The following books are collections of papers:

J. Anderson & G. Hinton:  
Parallel Models of Associative Memory. Erlbaum (1981)

S. Amari & M. A. Arbib: 
Competition and Cooperation in Neural Nets.  Springer (1982)

W. B. Levy, J. A. Anderson, S. Lehmkuhle:
Synaptic Modification, Neuron Selectivity, and Nervous System
Organization.  Erlbaum (1984)

J. L. McClelland & D. E. Rumelhart:
Parallel Distributed Processing: Explorations in the Microstructure of 
Cognition.  MIT Press (in press)

Between their content and their bibliographies, the above books should
direct to most of the important work in this field.  But let me note a
few articles for you also:

Kohonen, T., Oja, E.: Fast adaptive formation of orthogonalizing filters
and associative memory in recurrent networks of neuron-like elements.
Biol. Cybern. Vol 21: 85-95 (1976)

Sutton, R. S., Barto, A. G.: Toward a modern theory of adaptive networks:
expectation and prediction.  Psych. Rev. Vol. 88: 135-170 (1981)

Miyake, S., Fukushima, K.: A neural network model for the mechanism of