Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!utgpu!water!watmath!clyde!rutgers!ames!amdahl!rtech!daveb
From: daveb@rtech.UUCP
Newsgroups: comp.arch,comp.org.usenix
Subject: Re: Benchmarking the 532, 68030, MIPS, 386...at a Usenix!
Message-ID: <826@rtech.UUCP>
Date: Fri, 15-May-87 03:34:14 EDT
Article-I.D.: rtech.826
Posted: Fri May 15 03:34:14 1987
Date-Received: Sat, 16-May-87 15:06:45 EDT
References: <324@dumbo.UUCP> <809@killer.UUCP> <2417@homxa.UUCP> <4294@nsc.nsc.com> <2128@hoptoad.uucp>
Reply-To: daveb@rtech.UUCP (Dave Brower)
Organization: Relational Technology, Alameda CA
Lines: 44
Xref: utgpu comp.arch:1196 comp.org.usenix:156
Summary: Do vendors really *want* to be benchmarked like this?

In article <2128@hoptoad.uucp> gnu@hoptoad.uucp (John Gilmore) writes:
>
>	Let's have the bake-off in the trade show at, say, next Winter
>	Usenix.  Probably the actual setup and running of the benchmarks
>	can be done a day or two before the show, so the results can be
>	printed for distribution, and to give the losers time to think
>	up (and print up) good explanations before we descend on them :-).
>
>	Let's also make the same setup of machines available for people
>	to run their own benchmarks...

At last winter's Uniforum, I went around to a number of booths trying to
run the infamous

	/bin/time bc << !
	2^4096
	!

At a distressing number of places the sales creatures in the booth would
say things like, "I don't believe we're interested in running any
benchmarks today.  Let me show you vi."  Now there are some good reasons
for this, but it sure sounded like there was something being hidden.

Problem 1 is getting some benchmarks run.  Problem 2 is trying to get a
straight answer on the price of the system.  What you really want is the
bang/buck of different benchmarks on different boxes.  The results would
be an embarrassing to many people wearing suits, which is why it may be
difficulty to get a lot of cooperation.

-dB

PS:  Given my druthers, I'd like to see:

	* the bc benchmark above
	* Dhrystone
	* Whetstones
	* A paging thrasher.
	* A system call overhead checker (looped getpid()s maybe).
	* A process thrasher.

I'd probably give up on disk speed and tty i/o.

-- 
{amdahl, cbosgd, mtxinu, ptsfa, sun}!rtech!daveb daveb@rtech.uucp