Path: utzoo!attcan!utgpu!news-server.csri.toronto.edu!rutgers!usc!snorkelwacker.mit.edu!bloom-beacon!eru!hagbard!sunic!news.funet.fi!ousrvr!tko.vtt.fi!dfo
From: dfo@tko.vtt.fi (Foxvog Douglas)
Newsgroups: comp.ai
Subject: Re: How much info can the brain hold?
Message-ID: <1990Dec19.094328.24247@ousrvr.oulu.fi>
Date: 19 Dec 90 09:43:28 GMT
References: <11941@hubcap.clemson.edu> <61534@bbn.BBN.COM>
Sender: news@ousrvr.oulu.fi
Organization: Technical Research Centre of Finland, Computer Laboratory
Lines: 90

In article <61534@bbn.BBN.COM> syswerda@labs-n.bbn.com (Gilbert Syswerda) writes:
>Jacob Schwartz estimates the capacity of the human brain in an article in
>the Winter 1988 Daedalus. It is an interesting article, with estimates
>based on brain physiology. 

>  "...Such exceedingly rough quantitative guesses lead us to estimate that
>the long-term memory available to the brain is about 10,000 trillion bytes
>and that the amount of shorter-term data needed to characterize the state
>of each of its synapses is roughly the same. 

This number is unbelievable.

>The logical activity of each
>neuron can then be regarded as a process that combines approximately 10
>thousand input 

Of many kinds of neurons, most have far less than 10,000 input synapses.

>bytes

Why assume that 8 bits of data are transmitted at any instant instead of
up to 2 bits?
 
>with roughly 40 thousand synapse status bytes 

This implies 4 states per synapse (with all synapses independent) = 2 bits.

>at a rate of 100 times each second. 

Is arithmetic done for each 100th of a second?  Aren't many signals
pulse trains in which frequency is important?

>The amount of analog arithmetic required for

I dare say that the 4*10^4 operations simultaneously
occurring in a neuron are not independent.  The neuron may activate
depending upon the number of positive inputs (often with a cutoff
function) with the possibility of some inputs alternatively forcing
the neuron either on or off.  You could impliment this in an analog
computer with simple components for each synapse and minimal other
computational resources.  Using digital logic the number of steps would
depend upon the fan in.

Why not use 100 ic's with a fan-in of about 100 with a few bits output
tied into another such ic.  One such array running at 200 Hz would
emulate a neuron's input & processing (a bus would handle output).  Run
it at 20 MHz and it would emulate 10^5 neurons.  (And therefore process
10^12 elementary operations per second (see below))

>this estimate is (again very roughly) 10 million elementary operations per
>neuron per second,      ^^^^^^^^^^^^

40,000 * 100 approx.= 10,000,000 ?
This seems to suggest that a single neuron has a computing speed higher
than cheap PCs (ignoring the fact that output is at most 100 bits
(bytes???) per second (although widely branched).

>suggesting that the computing rate needed to emulate the
>entire brain on a neuron-by-neuron basis 
                   ^^^^^^^^^^^^^^^^
Why not emulate on a molecule-by-molecule basis , you'll get a higher
number.-)

>may be as high as 1,000,000
>trillion arithmetic operations per second. 

Assuming 100 billion neurons each of which can perform 10^7 operations
per second.  

>(Of course, computation rates many orders of magnitude 
                               ^^^^^^^^^^^^^^^^^^^^^^^^
>lower might suffice to represent the logical
>content of the brain's activity if it could be discovered what this is.)

>   It is interesting to compare these exceedingly coarse estimates with
>corresponding figures for the largest supercomputer systems likely to be
>developed over the next decade. These will probably not attain speeds in
>excess of 1 trillion arithmetic operations per second, which is about
>one-millionth of the computation rate that we have estimated for the
>^^^^^^^^^^^^^      brain."

How does "many orders of magnitude" compare with one million?

It seems to me that even these high figures for computational speed of
the brain, when compensated for as suggested in the parenthetical
expression, are not vastly beyond what can be achieved in hardware in
the not too distant future.  

doug foxvog
dfo@vtt.fi