Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!iuvax!cica!gatech!hubcap!brian
From: brian@june.cs.washington.edu (Brian Bershad)
Newsgroups: comp.parallel
Subject: Re: Looking for a message  passing package to run on the Sun
Message-ID: <6496@hubcap.clemson.edu>
Date: 18 Sep 89 18:22:53 GMT
Lines: 48
Approved: parallel@hubcap.clemson.edu

avsd!childers@decwrl.dec.com (Richard Childers) writes:

>I recall reading about a remote execution daemon that had been developed
>experimentally a few years ago ... at UC Berkeley.

>I recall reading about it in the USENIX journal, oh, about three years ago.

>I'll bet a few hours skimming through old USENIX journals might benefit
>you. As I recall, the daemon, when given a job, polled the network to see
>what machines weren't busy and passed the job out to appropriate servers,
>also running this daemon. When they were done, it was returned.

>I think it didn't fly because the overhead was too high, networking and on
>the machines that ran the daemon, to make it cost-effective in the environment
>it was conceived in. But it would still seem like the ideal core for a loose
>approach to multiprocessing, provided you had a front end that was capable
>of dividing the task up into independent pieces that could be processed w/o
>dependencies ....

The system was called Maitrd and it 'flew' quite well.  I think it is still
in use on a cluster of 750's and a 785 at Berkeley for balancing compiles
and text formatting.

The majority of the overhead came from not having any kind of network file
system.  File preprocessing was done locally and shipped to whichever server
was currently available.  For C programs, this was easy.  It was a bit hairier
for pascal code, which had intermodule type checking (with files being treated
as types).

There was almost no overhead to running the daemons.  Servers decided whether
they were willing to accept or not, and that information was only broadcast
when it changed.  Clients round-robined the jobs off to the available servers
only when their own load was above threshold.  

The biggest drawback was the effort required to build a new remote service
which had to act as a shell for the job on the remote end.    But, since
there were only a few tasks that had to be load-balanced,  the one-time
investment was worth it.  

RPC would not have made this any easier.


A paper showed up in a Usenix newsletter in early 1986: 
	"Load Balancing With Maitrd"

People have been doing this kind of stuff for years.

    Brian