Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!wuarchive!udel!rochester!pt.cs.cmu.edu!gandalf.cs.cmu.edu!lindsay
From: lindsay@gandalf.cs.cmu.edu (Donald Lindsay)
Newsgroups: comp.arch
Subject: Re: Networking for Distributed Computing
Message-ID: <12606@pt.cs.cmu.edu>
Date: 7 Apr 91 17:50:25 GMT
References: <1991Apr5.182853.20728@hubcap.clemson.edu>
Organization: Carnegie Mellon
Lines: 35

In article <1991Apr5.182853.20728@hubcap.clemson.edu> mccalpin@perelandra.cms.udel.edu (John D. McCalpin) writes:
>Unfortunately, the most commonly available networking
>option (ethernet) uses a broadcast approach, which is definitely
>sub-optimal for the communications needs of many "natural"
>parallel distributed algorithms.

Actually, shared-channel broadcast is optimal, as long as
- you don't run out of bandwidth (or pile up big latencies).
- detecting that a message isn't for you, doesn't impact your performance.

The trouble with point-to-point links is that you wind up implementing
message forwarding, the downside being
- more code
- more latency
- performance impact on the intermediaries

You can sort-of cure this with the right topology, or with hardware
support, but you still have to write the code or design the hardware.
(There have been some very interesting attempts lately - e.g. DEC SRC
AutoNet - to build systems which dyamically discover their topology.)

>In talking to my IBM salescritter about this topic, he suggested using
>an additional SCSI interface as the networking option.

>A suitably designed code (for example a 3-D spectral element code for
>fluid dynamics using explicit time marching techniques) should be
>capable of 1 GFLOPS performance on a network of 32 IBM RS/6000-320's.

Kung's "Law" says that if you scale node performance, without
increasing communication bandwidth, then nodes require more memory:
N, N^2 or even N^3 as much, depending on algorithm. Before choosing a
communications setup, I would want to study your application's
characteristics, and work up some ratios and granularities.
-- 
Don		D.C.Lindsay .. temporarily at Carnegie Mellon Robotics