Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!linus!decvax!harpo!gummo!whuxlb!floyd!clyde!ihnp4!zehntel!hplabs!sri-unix!rik@UCLA-CS
From: rik@UCLA-CS@sri-unix.UUCP
Newsgroups: net.ai
Subject: Parallelism and Physiology
Message-ID: <12283@sri-arpa.UUCP>
Date: Fri, 30-Sep-83 17:45:55 EDT
Article-I.D.: sri-arpa.12283
Posted: Fri Sep 30 17:45:55 1983
Date-Received: Thu, 6-Oct-83 03:41:07 EDT
Lines: 109

From:  Rik Verstraete <rik@UCLA-CS>

I would like to comment on your message that was printed in AIList Digest
V1#63, and I hope you don't mind if I send a copy to the discussion list
"self-organization" as well.

        Date: 23 Sep 1983 0043-PDT
        From: FC01@USC-ECL
        Subject: Parallelism

        I thought I might point out that virtually no machine built in the
        last 20 years is actually lacking in parallelism. In reality, just as
        the brain has many neurons firing at any given time, computers have
        many transistors switching at any given time. Just as the cerebellum
        is able to maintain balance without the higher brain functions in the
        cerebrum explicitly controlling the IO, most current computers have IO
        controllers capable of handling IO while the CPU does other things.

The issue here is granularity, as discussed in general terms by E. Harth
("On the Spontaneous Emergence of Neuronal Schemata," pp. 286-294 in
"Competition and Cooperation in Neural Nets," S. Amari and M.A. Arbib
(eds), Springer-Verlag, 1982, Lecture Notes in Biomathematics # 45).  I
certainly recommend his paper.  I quote:

One distinguishing characteristic of the nervous system is
thus the virtually continuous range of scales of tightly
intermeshed mechanisms reaching from the macroscopic to the
molecular level and beyond.  There are no meaningless gaps
of just matter.

I think Harth has a point, and applying his ideas to the issue of parallel
versus sequential clarifies some aspects.

The human brain seems to be parallel at ALL levels.  Not only is a large
number of neurons firing at the same time, but also groups of neurons,
groups of groups of neurons, etc. are active in parallel at any time.  The
whole neural network is a totally parallel structure, at all levels.

You pointed out (correctly) that in modern electronic computers a large
number of gates are "working" in parallel on a tiny piece of the problem,
and that also I/O and CPU run in parallel (some systems even have more than
one CPU).  However, the CPU itself is a finite state machine, meaning it
operates as a time-sequence of small steps.  This level is inherently
sequential.  It therefore looks like there's a discontinuity between the
gate level and the CPU/IO level.

I would even extend this idea to machine learning, although I'm largely
speculating now.  I have the impression that brains not only WORK in
parallel at all levels of granularity, but also LEARN in that way.  Some
computers have implemented a form of learning, but it is almost exclusively
at a very high level (most current AI on learning work is at this level),
or only at a very low level (cf. Perceptron).  A spectrum of adaptation is
needed.

Maybe the distinction between the words learning and self-organization is
only a matter of granularity too. (??)

        Just as people have faster short term memory than long term memory but
        less of it, computers have faster short term memory than long term
        memory and use less of it. These are all results of cost/benefit
        tradeoffs for each implementation, just as I presume our brains and
        bodies are.

I'm sure most people will agree that brains do not have separate memory
neurons and processing neurons or modules (or even groups of neurons).
Memory and processing is completely integrated in a human brain.
Certainly, there are not physically two types of memories, LTM and STM.
The concept of LTM/STM is only a paradigm (no doubt a very useful one), but
when it comes to implementing the concept, there is a large discrepancy
between brains and machines.

        Don't be so fast to think that real computer designers are
        ignorant of physiology.

Indeed, a lot of people I know in Computer Science do have some idea of
physiology.  (I am a CS major with some background in neurophysiology.)
Furthermore, much of the early CS emerged from neurophysiology, and was an
explicit attempt to build artificial brains (at a hardware/gate level).
However, although "real computer designers" may not be ignorant of
physiology, it doesn't mean that they actually manage to implement all the
concepts they know.  We still have a long way to go before we have
artificial brains...

        The trend towards parallelism now is more like
        the human social system of having a company work on a problem. Many
        brains, each talking to each other when they have questions or
        results, each working on different aspects of a problem. Some people
        have breakdowns, but the organization keeps going. Eventually it comes
        up with a product, although it may not really solve the problem posed
        at the beginning, it may have solved a related problem or found a
        better problem to solve.

Again, working in parallel at this level doesn't mean everything is
parallel.

                Another copyrighted excerpt from my not yet finished book on
        computer engineering modified for the network bboards, I am ever
        yours,
                                                Fred


All comments welcome.

        Rik Verstraete <rik@UCLA-CS>

PS: It may sound like I am convinced that parallelism is the only way to
go.  Parallelism is indeed very important, but still, I believe sequential
processing plays an important role too, even in brains.  But that's a
different issue...