Xref: utzoo comp.ai.philosophy:777 comp.ai.neural-nets:3119 Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!uupsi!sunic!news.funet.fi!polaris.utu.fi!magi From: magi@utu.fi (Marko Gronroos) Newsgroups: comp.ai.philosophy,comp.ai.neural-nets Subject: Re: Continuous vs. discrete Message-ID: Date: 28 Mar 91 17:37:10 GMT References: <91082.223501DOCTORJ@SLACVM.SLAC.STANFORD.EDU> <1991Mar25.141743.21124@news.larc.nasa.gov> <1991Mar26.215728.28875@watserv1.waterloo.edu> Sender: usenet@polaris.utu.fi (Usenet News) Organization: University of Turku, Finland Lines: 121 In-Reply-To: ssingh@watserv1.waterloo.edu's message of 26 Mar 91 21:57:28 GMT (Any information about research on this subject would be appreciated.) ssingh@watserv1.waterloo.edu (Sneaky Sanj ;-) said: >The mind MUST be discrete. It is a quantum-mechanical machine. This MUST? Interesting. I suppose it's O.K. that energy quantities and therefore the matter quantities may be discrete, but I'm not too familiar with quantum physics, so it sounds suspicious that time and space would be discrete too. I don't think that the differences between quantum and conventional physics would affect something so complex as the brain, at least not significantly (hopefully). Nice idea from religious point of view... If someone could prove that the space and time are discrete, one might speculate that we are living in a computer simulation. :-) (If the God doesn't know how to build continuous computers, how could we... 8->) But I don't think that this was the meaning of the original artical (was it?). Many current connectionist theories assume that the brain can be simulated with synchronized and discrete (sometimes even binary!) in time and space and quantity computers. I think my chessboard - ice hockey example shows this problem quite clearly. Has anyone done any research on this? I don't know too many neural network theories that include for instance temporal summation even in iterative neurons. Does someone disagree with these definitions (or have these been defined earlier somewhere? In some other way?): (virtually) continuous-in-time (or space) simulation = simulation in continuous time/space (impossible with modern computers) or with a (small) fixed time/space step, for instance 1 millisecond/micrometer (possible with computers, but slow). Simulation in (virtually) continuous space would mean that network structures have a "physical" shape. Iterative/synchronized-in-time simulation = Simulation with an abstract time-step where all operations are synchronized and take the same time (currently very common). Continuous-in-distance simulation= Neurons have an abstract size (null) but are located in at least virtually continuous space. Continuous or discrete or binary quantity = If weights/activation levels/action potentials can have values like [0,1] (cont.) or {0.0, 0.1, 0.2, ... 0.9, 1.0} (discr.) or {0, 1} (bin.). Structured(??) neurons (discrete-in-space??) = Neurons are divided in several parts (compartments/branches/sites). The real world is continuous in time, space, quantity and distance (maybe not in molecular level). Yes, action potentials are binary in quantity, but not in time/space... It makes me vomit when someone says "Hey! The brain is actually binary, like a computer", so don't be amazed if I react too strongly in this. Also, A.P.'s are not necessarily as binary as they seem to be. We must remember that AP's are just local ion levels, and they are quite different in different parts of neurons. The activation spreads everywhere in the neuron, not just in the some part. I don't know if this activation can cause any reactions, like releasing some neurotransmitter, even in a lower scale. There might also be 'micro-action-potentials'; if you inhibit the root of some dendritic branch strongly and exhibit the upper parts of the branch, it might generate an A.P. only in the branch and THEN be able to jump over the inhibitory area. Neurons within neurons? Why not? Any support on this? > Could someone tell me if there is any significant difference regarding the > properties of neural networks with a finite set of states for connection > strengths as opposed to continuous values. Which is more biologically > accurate? Depends on how many states there are in your finite set of states. 10? 1E10? 1E1000? 10 _stored_ states might be enough if you add some RND(). :-) Scaling is another problem. A synaptic weight can be 1 units and 10000 units. How about using short floating point numbers? 4 bits for mantissa and 4 bits for the exponent and 8 bit random number should be enough.. > I always thought that neurons assume one of a finite set of strengths. It > is just that it is a very large set, so from our vantage point it > appears continuous. I would like to explore the dynamical properties of > nonlinear neural networks, so this is important. Yes, the difficulty might come in changing the weights. The difference between 5 and 6 weights is not important, but changing them may be difficult. How about using RND() in that too - to change or not to change?? (like AP's in stochastic nets - to initiate or not to initiate, that is the question). Arun (????) writes in propably some very old article: >>binary float representation. I don't remember having seen any numbers, >>but I would tend to think that if you need more than 8 bits of >>resolution to get a neural computational model to work, the biological >>plausibility of such a model is suspect. I'd expect that there might rise some problems in some type of competetive learning when two neurons are competing for the representation of two patterns. If the two activation values are equal, and the learning algorithm is poor, the both neurons will represent both patterns. (that's just one example, but it gives some picture about what kind of problems there might be with discrete values). >>This is not to say, of course, that biological plausibility should be >>the acid test in evaluating models, especially application-oriented work. Yeps, but that's only for people who don't care a f*ck about science. They are the same people who think that there is nothing special if a computer can recognize handwritten text or speech like K.I.T.T. does (still remember Knight Rider?), or that neural nets are just a new batch of computers/applications that will help them in getting money (which unfortunately may be true, though). ------------------------------------------------------------------------------- Marko Gronroos ! Tel. +358-21-445613 ! Karvataskunkatu 10 H 100 ! ! Computer Scientists do it 20610 Turku ! ! with bigger hardware. Finland ! ! ------------------------------------------------------------------------------ Disclaimer: I am not responsible in anything that I do or write since my brain are controlling my actions ruthlessy. I have tried to sue my brain becouse of mental violence, but the policemen couldn't put it in handcuffs.