Xref: utzoo comp.unix.questions:26982 comp.unix.wizards:23809 comp.unix.admin:553 comp.unix.large:205 comp.unix.misc:579 comp.arch:19363
Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sdd.hp.com!wuarchive!udel!rochester!pt.cs.cmu.edu!o.gp.cs.cmu.edu!andrew.cmu.edu!zs01+
From: zs01+@andrew.cmu.edu (Zalman Stern)
Newsgroups: comp.unix.questions,comp.unix.wizards,comp.unix.admin,comp.unix.large,comp.unix.misc,comp.arch
Subject: Re: Killer Micro Question
Message-ID: <8bFAXS_00asB8UqAk3@andrew.cmu.edu>
Date: 17 Nov 90 05:21:34 GMT
References: <16364@s.ms.uky.edu>
Organization: Information Technology Center, Carnegie Mellon, Pittsburgh, PA
Lines: 75
In-Reply-To: <16364@s.ms.uky.edu>

randy@ms.uky.edu (Randy Appleton) writes:
> I have been wondering how hard it would be to set up several 
> of the new fast workstations as one big Mainframe.  For instance,
> imagine some SPARCstations/DECstations set up in a row, and called
> compute servers.  Each one could handle several users editing/compiling/
> debugging on glass TTY's, or maybe one user running X.

This has been done with mostly "off the shelf" technology at Carnegie
Mellon. The "UNIX server" project consists of 10 VAXstation 3100's (CVAX
processors) with a reasonable amount of disk and memory. These are provided
as a shared resource to the Andrew community (three thousand or more users
depending on who you talk to). The only other UNIX systems available to
Andrew users are single login workstations. (That is, there is nothing
resembling a UNIX mainframe in the system.)

> 
> But how does each user, who is about to log in, know which machine to
> log into?  He ought to log into the one with the lowest load average, yet
> without logging on cannot determine which one that is.

For the UNIX servers, there is code in the local version of named (the
domain name server) which returns the IP address of the least loaded server
when asked to resolve the hostname "unix.andrew.cmu.edu". (The servers are
named unix1 through unix10 .) I believe a special protocol was developed
for named to collect load statistics but I'm not sure. As I recall, the
protocol sends information about the CPU load, number of users, and virtual
memory statistics.

Note that all asynch and dialup lines go through terminal concentrators
(Annex boxes) onto the CMU internet.

> [Stuff deleted.]
> ...  Also, the login server could have 
> all the disks, and the others would mount them, so that no matter what node
> you got (which ought to be invisible) you saw the same files.

Andrew uses Transarc's AFS 3.0 distributed filesystem product to provide
location transparent access to files from any workstation or UNIX server in
the system. There are other problems which are solved via AFS components as
well.  For example, traditional user name/password lookup mechanisms fail
badly when given a database of 10,000 registered users. AFS provides a
mechanism called the White Pages for dealing with this. (Under BSD UNIX,
one can use dbm based passwd files instead.)

If you want more info on the UNIX server project, send me mail and I'll put
you in touch with the appropriate people. Detailed statistics are kept on
the usage of these machines. Using these numbers, one could probably do
some interesting cost/performance analysis.

> 
> The advantage is that this seup should be able to deliver a fair 
> number of MIPS to a large number of users at very low cost.  Ten SPARC
> servers results in a 100 MIPS machine (give or take) and at University
> pricing, is only about $30,000 (plus disks).  Compare that to the price
> of a comparabe sized IBM or DEC!
> 
> So my questions is, do you think this would work?  How well do you
> think this would work?  Do you think the network delays would be
> excessive?

Yes, what you describe could be easily done with ten SPARCstations and a
small amount of software support. It is not clear that it is useful to
compare the performance of such a system to that of a mainframe though. It
depends a lot on what the workload is like. Also, there are other points in
the problem space. Three interesting ones are tightly coupled
multi-processors (SGI, Encore, Pyramid, Sequent, Solbourne), larger UNIX
server boxes (MIPS 3260s or 6280s, IBM RS/6000s, faster SPARCs, HP, etc.),
and 386/486 architecture commodity technology (PC clones, COMPAQ
SystemPRO). Certainly, DEC VAXen and IBM 370s do not provide cost effective
UNIX cycles but that is not the market for that type of machine.  Intuition
tells me that the best solution is very dependent on your workload and the
specific prices for different systems.

Zalman Stern, MIPS Computer Systems, 928 E. Arques 1-03, Sunnyvale, CA 94086
zalman@mips.com OR {ames,decwrl,prls,pyramid}!mips!zalman