Path: utzoo!attcan!uunet!zephyr.ens.tek.com!tektronix!sequent!mntgfx!msellers
From: msellers@mentor.com (Mike Sellers)
Newsgroups: comp.ai.neural-nets
Subject: Re: Postulates on the number of neurons
Message-ID: <1990Dec12.181155.2542@mentor.com>
Date: 12 Dec 90 18:11:55 GMT
References: <1990Dec11.174921.26838@cs.columbia.edu> <1990Dec11.040646.20760@noose.ecn.purdue.edu> <11678@ccncsu.ColoState.EDU>
Organization: engr
Lines: 186

In article <11678@ccncsu.ColoState.EDU> ld231782@longs.LANCE.ColoState.EDU (Lawrence Detweiler) writes:
>
>muttiah@welsh.ecn.purdue.edu (Ranjan S Muttiah) in
><1990Dec11.040646.20760@noose.ecn.purdue.edu :
>
>>What do researches think about why there are so many _individual_ neurons
>>in the human brain ... ?  Is there some critical number
>>needed for "intelligent behaviour" ?
>
>esrmm@warwick.ac.uk (Denis Anthony) replies in
><1990Dec11.152350.16930@warwick.ac.uk>:
>
>>The speed of processing of the nervous system (milliseconds) and the amount
>>of computation required for (e.g.) vision in real time, and the number of
>>simultanious computation systems (vision, sound, proprioception, cognition
>>etc.) require a large number of parallel processes. (all pretty obvious).

Feldman articulated this in his "100-step" rule.  Since neurons are slow,
operating in terms of milliseconds rathern than nano-seconds, and yet we 
are able to perform significant feats of cognition in a relatively short 
time frame (< 1 second), only about 100 sequential processing steps could 
fit into the time needed to perform the sensation, perception, and cognition 
of, say, recognizing a written word.  Since 100 sequential steps are
nowhere near what is necessary to do this sort of thing, massive parallelism 
must be used.  Thus a large number of neurons, each with a large number 
of interconnections, is required for fast processing with slow units.
(The number of neurons probably is something like 10^11, though this is
obviously a _very_ gross estimate, and the total number of synapses --where
the processing probably takes place-- is something like 4 to 6 orders of
magnitude greater than the number of neurons.)

> [...] Biology, in all its grandeur, is rarely a brilliant technological
>innovator in the eyes of one of its offspring, those sole thinking critics

I would argue this point, but that is probably best left for elsehwere.

>(after all, it hasn't even invented and applied the *wheel* in any of its
>abundant creations and variations.)  

Not so.  At least some flagellate bacteria drive their flagella in a 
circular, not back-and-forth type motion via a small "wheel" at the 
base of the flagellum.  At this scale having a proteinaceous wheel
structure seems to make sense; at the macroscopic scale it apparently
doesn't (I have an article around somewhere about why animals don't
use wheels -- from SciAm or Discover or something, I think).

> Parallelism is an unmistakably intrinsic
>element in biology, however.  It is the secret of life's success.  Even though
>single-celled organisms (bacteria etc.) probably outnumber the multi-celled
>ones in our realm, I would wager the vast majority of cells are contained in
>multicellular organisms. 

I realize that this is another side-comment, but considering the huge amount
of the earth's biomass that is held in bacteria, algae, and other unicellular
organisms, I think multicelled organisms are almost certainly in the minority.
And anyway, the survival strategy of being multicellular is more an argument 
for object-oriented paradigms than parallel ones. :-)

>  So biology has had some bias for parallelism for
>some time, and we have something to learn from our patient and pragmatic
>engineer.
>
>I'm sure there are many opinions on what comprises the "critical mass" in
>human brain tissue that separates them so distinctly from the rest of the
>animal kingdom.  
  
I'm not so sure a distinct separation between human cognitive abilities
and those of all other animals exists.  The cognitive abilities of some
of the great apes have been demonstrated, as have those of some cetaceans.
We do not (yet?) have enough of a common basis upon which to judge their
neural/cognitive abilities in comparison to ours, but I think it is safe
to say that the traditional view of a chasm separating "man and beast"
has been replaced with the realization that there are great but gradual 
changes in neural, perceptual, and cognitive abilities from planaria to
humans.

> One possible candidate is the huge amount of "interneurons"
>in the human brain.  Neuroscience found a distinction of motor- vs. sensory-
>vs. inter- neuron useful in categorizing the nervous systems of invertebrates.
>Motor neurons connect to muscle fibers, or nearly so, and sensory neurons
>connect to receptors.  Unfortunately, as we climb the vertebrate ladder, so to
>speak, this distinction becomes increasingly meaningless, whereupon in homo
>sapiens we reach a climax where only one in many thousands of neurons can be
>labelled a "sensory" or "motor" neuron, and the vast remaining majority take
>on that nebulous and mysterous term "interneurons".  Therein lies thought,
>somewhere.  Some might balk at the idea of some "critical mass" of neuron
>number whereupon we reach an explosion of intelligence.  However, the idea is
>not so absurd to dismiss without thought...our brains have changed remarkably
>little in terms of organization in comparison with other invertebrates.  

I agree and disagree. :-)  I agree that "thought" is embodied (if that terms
makes any sense in this case) in the number and configuration of interneurons,
particularly those in the cerebral cortex.  However, I do not think it is 
correct to say that our brains have changed little from our invertebrate 
ancestors (or contemporaries).  Even among vertebrates there are major 
differences in large-scale neural structures; it isn't the case that human 
brains just have an extra lobe tacked onto them.

>What about this "critical mass" of neurons?
>
>>Clearly there is a minimum number, [the order] I do not know. There is some
>>redundancy in that fairly large portions of the cerebrum may be lost before
>>gross brain damage is manifest (though other parts of the brain are
>less robust).
>
>There is a long history of misunderstanding the inherent quality of
>fault-tolerance in a massively parallel system.  Reports of its nature have
>been greatly misinterpreted.  Because we remove nine-tenths of a rat's brain
>and it still navigates a learned maze with minimal impairment, does that imply
>that nine-tenths of our mental capacity is dormant?  We are simply applying a
>concept where it is not appropriate.  In truth, this concept of "graceful
>degradation" is a somewhat novel concept to human engineers.  There aren't too
>many engines that gradually slow down as we remove pistons, or buildings that
>gradually lean while we remove columns!  Just how crucial is one neuron to a
>thousand?  One one-thousandth!

I agree that the case for graceful degradation in neural systems is often
overstated.  However, it is true that many people who have massive cortical
damage rarely or never exhibit changed behavior.  There are other cases 
where less severe damage has resulted in spectacular cognitive or behavioral
deficits, so the situation is not a clear-cut one.  It is not the case,
however, that if you remove some fraction of the neurons from a neural
system that that system will necessarily be impaired to a similar degree.  

>minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky) in
><4311@media-lab.MEDIA.MIT.EDU>
>
>>It would seem likely, at least to me, that most single cells don't do
>>useful operations alone, so that, for example, it might need a whole
>>cluster or column of cells to do some little useful job.  So for many
>>functions, you'd be left with "only" a few hundred million" "units".
>>And it would seem that an ordinary "commonsense database" semantic
>>network might well require a few dozen million nodes and connections.
>>No one knows that yet, of course.
>
>Every neuron is doing a useful job, its just that we have a difficult time
>measuring/quantifying/characterizing its diffuse and miniscule contribution. 

I think you're using the term "useful job" in a different sense, or at a
different scale.  If you define "recognizing a vertical bar" or "recognizing
my grandmother" as useful jobs or operations, then almost certainly single 
neurons do not perform useful operations.  If you define reporting the
existance of light in a particular part of the visual field as a useful
job, then at least in some cases single neurons are doing useful things.
The point of arguments like Minsky's is that it may be that neither the 
neuron nor the synapse is the correct unit of perception and cognition,
but that ensembles of neurons provide more information for explanation 
and prediction of how thought is accomplished.

>fahn@cs.columbia.edu (Paul N. Fahn) in
><1990Dec11.174921.26838@cs.columbia.edu>
>
>>It seems to me that (computer-based) neural net researchers too often
>>presume that biological organization is somehow optimal. It may be that
>>the brain represents an enormous waste of resources, and that 
>>"intelligent behavior" can be achieved with far fewer neurons, if only
>>they were organized more "intelligently". (or, equivalently, that higher
>>intelligence could be achieved with the same number of neurons.)
>>
>>We should definitely look at biological nets for ideas, but in a very
>>critical way, and be prepared to reject anything found in the biological
>>realm.
>
>That's an interesting question that takes one step farther Marvin Minsky's
>favorite idea that neural networks can be condensed (ideally
>computationally)--not only can units be amalgamated, but some can be
>eliminated!  Is there a neurological equivalent of an appendix or tonsils? 
>Someday we may be surprised and aghast at the depth and vastness of the
>evolutionary baggage we carry...  But only when we understand the design we
>will have the opportunity to transcend it.
>
>ld231782@longs.LANCE.ColoState.EDU

I think we can look at evolution as a process of finding minima.  Our
brain structure representss a local minimum in terms of using space, 
energy, speed, etc..  There may well be a global minimum just over the 
next (silicon) hill, but we shouldn't be too quick to discard what a much
less innovative but far more tenacious process has come up with over
a period of several million years.


-- 
Mike Sellers     msellers@mentor.com     Mentor Graphics Corp.

"I used to think that the brain was the most wonderful organ in my
body.  Then I realized who was telling me this." -- Emo Phillips