Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!wuarchive!usc!ucsd!sdcc6!babymilo!pluto From: pluto@babymilo.ucsd.edu (Mark Plutowski) Newsgroups: comp.ai.neural-nets Subject: Re: Defining a Nerual Network Message-ID: Date: 19 Nov 90 19:55:37 GMT References: <2491@bimacs.BITNET> <6904@jhunix.HCF.JHU.EDU> <1990Nov18.140416.10297@lut.fi> Sender: news@sdcc6.ucsd.edu Lines: 79 Nntp-Posting-Host: babymilo.ucsd.edu slehar@thalamus.bu.edu (Steve Lehar) writes: >Let me try my hand at this one... >NEURAL NETWORKS >=============== >A neural network is a computational model that is inspired by >observation of natural computational mechanisms. Natural >architectures are fundamentally different from conventional >architectures in that they tend to represent information in a >distributed way, and to perform computation in a parallel analog >manner that seems to be more fault tolerant and robust if the input >information is somewhat ambiguous. Neural approaches work best in >applications where traditional computation has performed poorly, >usually because the data is ambiguous or the context has a large >influence on the data, such as vision, speech and cognition. They >generally perform poorly in realms where computers perform well, >usually because the data is deterministic, clearly defined and well >understood, such as word processors, spreadsheets, arithmetic >computation. I believe this is a decent characterization, and provides enough historical background to motivate the definition, but let me try my hand at a definition: "Recall the usual definition of a ``network'' as a connected graph of computing elements (nodes) in which communication among nodes occurs along arcs connecting the nodes (connections.) A ``neural'' network, by analogy with the biological namesake, is obtained by placing restrictions on the type of information allowed to propagate along the connections, as well as upon the type of computation allowed within each node. Each node is allowed to compute a mapping from a set of inputs to a scalar output value. The instantaneous value of the activation propagated by a connection is allowed to be a scalar value." Note that the inputs to a node can be elements of any set, and so this definition does not preclude symbolic input information, so long as the transfer function of the node is well-defined over such a domain. Usually, though, the inputs are assumed to be a vector of a real space, since so many learning algorithms are derived analytically. However, many learning algorithms are happy with inputs being elements of a set, since after all the two-bit input set {0, 1} can just as easily be taken to be the set {off, on} by slight modification of the learning algorithm, viz, by the appropriate use of propositions defined over the input set. The definition of what a connection is allowed to propagate does not preclude time-varying information, nor does it preclude propagation of symbolic information encoded as a scalar value, say, such that the symbolic information can be appropriately decoded at the other end. Also, the definition of network does not preclude global broadcasting of information, since we have placed no constraints upon the connectivity. Nor does it preclude recurrent dynamics, since each node may retain a history of previous inputs internally, or be served by an external set of nodes whose purpose it is to retain historical information, say, by emulating a stack, queue, or even a time-averaged statistical summary. It is important to define a neural network as a computational model, as Steve Lehar did above, since a neural network is not defined by its implementation, but as an abstract way of characterizing either a particular computational set of hardware, software, or, according to some folks, wetware. Improvements to my definition are welcomed, it may not be sufficiently general to encapsulate everyone's idea of a neural network. But it certainly encompasses my own understanding of the term, given the nets I've seen reported on in the literature. That's my opinion, what's yours? (Qualification: some "connectionists" may maintain that this definition is strictly contained within their definition of a "connectionist architecture." I would not argue with that viewpoint.) -=-= M.E. Plutowski, pluto%cs@ucsd.edu UCSD, Computer Science and Engineering 0114 9500 Gilman Drive La Jolla, California 92093-0114