Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!seismo!husc6!linus!philabs!steinmetz!davidsen
From: davidsen@steinmetz.steinmetz.UUCP (William E. Davidsen Jr)
Newsgroups: comp.arch,comp.org.usenix
Subject: Re: Benchmarking the 532, 68030, MIPS, 386...at a Usenix!
Message-ID: <6024@steinmetz.steinmetz.UUCP>
Date: Wed, 20-May-87 12:39:39 EDT
Article-I.D.: steinmet.6024
Posted: Wed May 20 12:39:39 1987
Date-Received: Sat, 23-May-87 09:39:48 EDT
References: <324@dumbo.UUCP> <809@killer.UUCP> <2417@homxa.UUCP> <15484@gatech.gatech.edu> <401@ralmar.UUCP>
Reply-To: davidsen@kbsvax.steinmetz.UUCP (William E. Davidsen Jr)
Organization: General Electric CRD, Schenectady, NY
Lines: 23
Xref: mnetor comp.arch:1368 comp.org.usenix:197

In article <401@ralmar.UUCP| ralph@ralmar.UUCP (Ralph Barker) writes:
|
|Assuming succes in obtaining manufacturer participation in the proposed
|"Bench-Off", what are the thoughts of using Neal Nelson's Business
|Benchmark(tm) for this purpose?  

As the developer of a benchmark suite of my own I would love to cast
bricks at the Nelson suite. In truth it's a pretty good set of benchmarks,
and has been run on hundreds of configurations. I agree that it would be
a suitable measure of machines.

After doing benchmarks for about 15 years now, I will assure everyone
that the hard part is not getting reproducable results, but in (a)
deciding how these relate to the problem you want to solve, and (b) getting
people to believe that there is no "one number" which can be used to
characterize performance. If pressed I use the reciprocal of the total
real time to run the suite. It's as good as any other voodoo number...

-- 
bill davidsen			sixhub \	ARPA: wedu@ge-crd.arpa
      ihnp4!seismo!rochester!steinmetz ->  crdos1!davidsen
				chinet /
"Stupidity, like virtue, is its own reward"