Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!cornell!uw-beaver!ubc-cs!alberta!calgary!ctycal!ingoldsb
From: ingoldsb@ctycal.UUCP (Terry Ingoldsby)
Newsgroups: comp.arch
Subject: Re: parallel systems
Summary: The problem with parallel systems
Message-ID: <499@ctycal.UUCP>
Date: 23 Oct 89 18:40:03 GMT
References: <35825@lll-winken.LLNL.GOV> <20336@princeton.Princeton.EDU> <7651@bunny.GTE.COM>
Organization: The City of Calgary, Ab
Lines: 44

In article <7651@bunny.GTE.COM>, hhd0@GTE.COM (Horace Dediu) writes:
> The only reason shared memory machines exist is because we don't yet know
> how to make good distributed machines.  (Yeah, right! tell that to Ncube) 
> IMHO shared memory is a hack using available bus technology while waiting for 
> the real parallel machines to come.  (they're already here)
....
> 
> Of course.  To solve hard problems you *need* parallel execution.
> It's no secret that every big iron maker and every supercomputer shop is
> developing parallel machines.  These are still modest efforts (<100 cpu's),
> but the leading egde is now in the 10k coarse grained, 64k fine grained
> processors.  This should scale nicely to 1M processors in the next decade.
> After that we can expect some kind of new barriers to come up.  

It is true that the only hope for the kind of performance improvement that will
be required for the next generation of software is parallel processing.  It is
also true that many different parallel processing machines exist.  Here is where
I start to be less optimistic about how quickly we can adopt parallel processing.

The basic problem is *NOT* the software.  Although awkward to write, it can be
done and it is reasonable to assume that techniques will be developed as the
years go by.  The (IMHO) fundamental problem is that different problems partition
differently.  By this I mean that to be executed across many P.E.s (processing
elements) a problem must be partitioned.  One criteria for partitioning the
problem is to subdivide it in such a way that the different P.E.s do as little
interP.E. communication as is possible.  Generally speaking, the more you divide
the problem, the more P.E.s need to talk to each other.  This, of itself, is
not necessarily bad (at least, it is tolerable).  The problem is that different
classes of problems subdivide in different ways.  Thus it is difficult to
set up any *efficient* communication strategy that will let P.E's talk to
other P.E.s (without going through lots of intermediary P.Es) for general
problems.  Until someone addresses this issue, I am hard pressed to believe
that a *general purpose* parallel machine will be developed.  I can easily
foresee a time when there will be a custom (parallel) vision processor in
a computer, a custom speech recognition processor, a custom database processor,
... and so on.  I cannot see a given parallel architecture doing all of the
above.

My 2 cents!

-- 
  Terry Ingoldsby                       ctycal!ingoldsb@calgary.UUCP
  Land Information Systems                           or
  The City of Calgary         ...{alberta,ubc-cs,utai}!calgary!ctycal!ingoldsb