Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!utgpu!water!watmath!clyde!cbosgd!ucbvax!husc6!bu-cs!bzs
From: bzs@bu-cs.UUCP
Newsgroups: comp.arch,comp.org.usenix
Subject: Re: Benchmarking the 532, 68030, MIPS, 386...at a Usenix!
Message-ID: <8080@bu-cs.BU.EDU>
Date: Wed, 27-May-87 19:50:37 EDT
Article-I.D.: bu-cs.8080
Posted: Wed May 27 19:50:37 1987
Date-Received: Fri, 29-May-87 01:26:43 EDT
References: <324@dumbo.UUCP> <809@killer.UUCP> <2417@homxa.UUCP> <15484@gatech.gatech.edu> <401@ralmar.UUCP> <2641@hcr.UUCP>
Organization: Boston U. Comp. Sci.
Lines: 48
Xref: utgpu comp.arch:1295 comp.org.usenix:196
In-reply-to: mike@hcr.UUCP's message of 20 May 87 22:59:51 GMT
Posting-Front-End: GNU Emacs 18.41.4 of Mon Mar 23 1987 on bu-cs (berkeley-unix)


Although I have nothing *against* a benchmark suite I still claim that
this is becoming less of an issue when compared against the richness
of the environment. Going full throttle for the flames, I wouldn't
trade my (mere :-) 2MIP Sun3/160 on (beside) my desk for a 10 MIP,
vanilla System V dumb terminal, no network, no job-control
environment, you'd have to pry the Sun out of my dead hands (tho I'd
take a 1 MIP SYSV over a 100MIP VMS system, it's all relative.)

I suspect I'm not alone in my opinion (not the particular systems, but
the idea that the quality of the software is beginning to outweigh
mere speed improvement.)

Think of it this way, I have an IBM3090/200 with two vector processors
(that's around 40MIPs and Cray-1S on floating point) at my disposal,
trust me, it's faster 'n hell, it's astoundingly fast, but the
software environment is so primitive I rarely log into it (and I
certainly know my way around an IBM system, it's not some fundamental
problem.) Oh, some number crunchers use it, and good for them, but boy
is that crowd getting relatively small (there are plenty of number
crunchers around here who would rather wait for their SUN3 or similar
box as far as I can tell.)

How we gonna measure that? I honestly think beyond some lower bounds
software is getting very important (and besides, they go hand in hand
to a great extent, you don't see too many window-oriented systems on
.5 MIP boxes, then again, the Mac comes close and I'd be happy argue
its virtues for getting one thru the day, we have those also, blows
the doors off the 3090 on people-performance for many daily tasks.)

And what about things like upgradeability (like my Encore that I can
jack up to around 40 (parallel) MIPs by just adding CPU boards)? I
know one major vendor who's only idea of an "upgrade" is you throw the
'old' $400K box away and buy a new $800K one...swell. Or a coherent
plan to spread the MIPs into the user's offices?

I still think there's a certain air of unreality to this whole "my
iron is bigger than your iron" thing. Oh, it's important, it's just no
longer a sufficient claim, certainly not enough to sell me on a box.

You can say "well, then assuming the two software environments are
equal..."  But they rarely are, often they're disasterously different
between two boxes.

I agree it's a much harder measure, but is that what we're after? The
cheap shots?

	-Barry Shein, Boston University