Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!uflorida!haven!aplcen!jhunix!ins_atge From: ins_atge@jhunix.HCF.JHU.EDU (Thomas G Edwards) Newsgroups: comp.ai.neural-nets Subject: Re: NN Question Summary: i/o and discrete time Keywords: discrete time Message-ID: <1023@jhunix.HCF.JHU.EDU> Date: 7 Mar 89 01:18:57 GMT References: <32125@gt-cmmsr.GATECH.EDU> <10624@pasteur.Berkeley.EDU> Reply-To: ins_atge@jhunix.UUCP (Thomas G Edwards) Distribution: usa Organization: The Johns Hopkins University - HCF Lines: 43 In article <10624@pasteur.Berkeley.EDU> brp@sim.UUCP (bruce raoul parnas) writes: > Neural nets take inputs and associate them with outputs, nothing more. They do not reflect even >the simplest levels of cognition! While it is definately true that we haven't even gotten anywhere close to a 10^13 neuron device like humans, one could very well argue brain is also a device which associates inputs, memory, and produces an output. Mind you, the tranfer function is very complex :-). Recurrent neural networks are capable of holding memories in neural "loops," and also there are algorithms for learning in a contiually running NN (Williams, Zisper "A Learning Algorithm for Continually Running Fully Recurrent Neural Networks" UCSD ICS 8805 Oct. 1988). >Since these models are, mainly, discrete time automata they do not reflect the fact that real neural systems are, essentially,nonlinear continuous-time multi-dimensional vector spaces in which the neurons >evolve in time. So while they are real neat computational tools, they are far >from representing real neural processes. Pineda, in "Dynamics and Architecture in Neural Compuation", Jorunal of Complexity, Sept. 1988, points out that time is very important to NN's, especially if we want to store multiple pattern associations in them. He proposes a formalization for recurrent NN's dealing with them as dynamical systems, and can thus bring them into continuous time instead of discrete time. (I think people working with recurrent nets should look at this paper...It didn't seem to draw the attention it deserved). Another major drawback to current neural networks is that human NN's are a product of evolutionary search. There are, however, a large bunch of people working with "neuro-evolution" now, and maybe we'll see some neat stuff. Also there is alot of neat recurrent stuff now which people who have only read PDP have missed out on. Someone needs to write a good book aimed at Joe Programmer concerning these issues (or has someone, and I have just missed it?) -Thomas Edwards ins_atge@jhuvms (BITNET) tedwards@nrl-cmf.arpa #include