Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!att!emory!hubcap!eugene
From: eugene@nas.nasa.gov (Eugene N. Miya)
Newsgroups: comp.parallel
Subject: Re: Critical Issues in Distributed-Memory Multicomputing.
Message-ID: <12235@hubcap.clemson.edu>
Date: 12 Dec 90 23:14:58 GMT
Sender: fpst@hubcap.clemson.edu
Reply-To: eugene@wilbur.nas.nasa.gov (Eugene N. Miya)
Organization: NAS Program, NASA Ames Research Center, Moffett Field, CA
Lines: 149
Approved: parallel@hubcap.clemson.edu

I go on vacation and come back to this....
In article <12051@hubcap.clemson.edu> gdburns@osc.edu writes:
>>
>>   The Distributed-Memory Multi-Computers have now been facilitated in 
>>many research institutions and Universities. The most exciting time of
>>...
>>for an endless variaty of applications. When could usual engineer
>>(not computer engineer) can use such machine with ease? 
>>
>>Jingwen Wang

A good post of some of the issues.

I think you have to ask if the usual engineer uses non-DMMCs with ease.
I assert not.  Read back in the literature using "automatic programming"
as a keyword.  It has gotten easier, but we have only learned with time
and experience.  We have a deceptive situation: some programming
appears very easy learn, the basics: syntax of programming languages for
simple programs, etc. 

>I am willing to take a shot at this.
>
>H/W is in reasonable shape.

A nice ATTEMPT at a state-of-the-art survey.
I don't quite agree.  We have perfected packaging, but we have not
perfected communication or storage.  We have a fetish for CPU speed
while ignoring balance.  Amdahl deserves a lot of credit for his little
3 page NCC paper (1967).

That you ignored difficulties with programming languages tells me we have
a long way to go.  A reviewer can tell the quality of a survey paper
with the %text on hardware versus the %software.  This also goes for
market literature, personnel in a start up, and a few other artifacts.

>from the vendors, in my opinion, because the customer tends to look at
>either the glamour of the h/w or the bottom line application.

One can't blame them for the bottom line especially when a technology
as incomplete as EE/CS exists.  A friend from IBM TJW (just
skiing with him) gave me an article which ended with:
"Creating the Universe was simple: God didn't have an installed based."

>OS doesn't
>get enough attention from academia because, IMHO, the work that needs
>to be done is grunt level - essentially big D and very little r.

The OS people are contrainted by the limitations of hardware.  End users
don't care as much about the OS.  That indifference creates some subtle
problems (Elxsi, Denelcor, and a few other defunct companies can
tell you about this.)  I had an interesting lunch discussion about
the concept of a "load average" on a multiprocessor.

>Algorithms are going well, I think, because of the number of people
>doing work.  Getting the work productized and distributed is not going well.
>I have not done a survey but I'm sure you could find a
>workable algorithm for every solver somewhere in the literature.

I do not think algorithms are going well.  I have done some survey.
Algorithms by their very nature tend to be sequential.  What tends to be
"parallel" is data.  There are a few books (Akl, Oretega and Voigt)
but these all tend to specialized algorithms.  I took the work "good"
out before "books."  It's too early to judge quality.  Akl's book is now
in second edition.  I don't know what you expect with "productization."
And there are algorithms which do not parallelize well as this time.
It depends on a vague concept of "level of parallelism."

>Education is the kicker.  You have to learn how to program distributed
>memory.  There are no true magic pills.

Very good (no silver bullets).

See you are re-inventing programming again.  Hence the automatic programming
comment.  This must have been what programming was like in 1946-1950.
Now, we replace the LOADs and STOREs with SENDs and RECEIVEs.
The goal started to hide the "underlying" architecture, but to get at the
speed, we need to.  It's no better with the FORALLs and DOALLs, or
host of other constructs.  The problem is not syntactic, it's
semantic.  It makes comparison and evaluation difficult.
We ended up creating something now called a compiler.  But we
are overloading the functionality of compilers.  Part of that is the
problem os existing programming languages built for a von Neumann
model of programming.

If the users are unwilling to explore new programming languages,
some models of parallel computation will be in trouble.  Perfect
automatic parallelization only exists in the mind of the media, hopeful
users, and very few computer scientists.

Anita Jones and others wrote some interesting comments about the ease
of parallel programming in the early 1980s.  Her reports and surveys
from CMU (Cm* and earlier C.mmp papers) noted many of the problems.  
We still have those problems today, yet we are not talking the same
human language.

We are so closed minded to other earlier approachs like LISP, APL,
and languages like VAL, SISAL, can't even get a chance for a reasonable
evaluation.

>Back to OS, I think that we need to get out our hammer and saw and
>build a quality lifestyle on multicomputers.  Quality to me means:
>
>1) a dynamic multi-tasking environment on every node
>2) a powerful, no-nonsense, save the pardigms for later, support everything
>   message passing system on every node
>3) Unix file access in all its glory from every node
>4) lots of control (user is boss not servant) and monitoring

Naive Motherhood: somewhat.  This is partially what led to CISC
architectures.  It also created Unix and RISCs. This isn't a flame, but
I think we put too much functionality into some parallel systems.
We have to simplify and strip out.  No one would serious consider
a new OS not written in a high level language (a development tar pit).
But we see people trying to explore concepts like light-weight
tasking.

>I think MIMDizer and F90 are logical next steps in multicomputer tool
>technology.  I/O and comm libraries like CrOS and Cubix are also very good.
>Linda and Strand are, IMHO, appropriate for functional parallelism.
>These things make multicomputer programming sensible and even fun.
>But not enough effort has been spent on open environments with process
>control/status, message status, error code delivery, and other mundane
>support whose absence can really ruin your day.

I am not so sure. (Sorry John 8^).  I don't know how best to describe
what we have to do.  Developer have to be able to pay around with
real (not simulated, we sometimes do too much of this, let's simlate
1 TFLOP......8^) ) architectural choices.  User communities must be
willing to try new developments like OSes and languages, and architecture.

We are all blind men trying to describe an elephant.  I recommend
Fred Brooks The Mythical Man-Month (1975). One thing Brooks noted was the
structure of a system reflects the bureaucracy which built it.
Parallel computing research is in a state of flux because of the immaturity
of the technologies which feed it (H/W and S/W).  It says something about
those doing research, development, and the funding agencies.  We
lack critical mass and we have a meek user community (the cost penalty is
high for a mistake).  This is why the "critical mass" analogy is important.
If we ignore or forget aspects of parallel computing, we will not see
problems or their solution.  We lack good conceptual models like the
von Neumann machine in sequential programming.

It is important in any science to have a health dose of skepticism.
CS/EE tends to be a bit weak on this (perenial optimists 8^).
A good test would be to guess what companies you would be willing to
invest personal, real dollars.  Or me for that matter.

--e. nobuo miya, NASA Ames Research Center, eugene@orville.nas.nasa.gov
  {uunet,mailrus,other gateways}!ames!eugene