Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!seismo!gatech!hao!ames!ucbcad!ucbvax!hoptoad!gnu
From: gnu@hoptoad.uucp (John Gilmore)
Newsgroups: comp.arch,comp.org.usenix
Subject: Benchmarking the 532, 68030, MIPS, 386...at a Usenix!
Message-ID: <2128@hoptoad.uucp>
Date: Thu, 14-May-87 21:59:40 EDT
Article-I.D.: hoptoad.2128
Posted: Thu May 14 21:59:40 1987
Date-Received: Sat, 16-May-87 13:13:03 EDT
References: <324@dumbo.UUCP> <809@killer.UUCP> <2417@homxa.UUCP> <4294@nsc.nsc.com>
Organization: Nebula Consultants in San Francisco
Lines: 56
Xref: mnetor comp.arch:1296 comp.org.usenix:172

In article <4294@nsc.nsc.com>, grenley@nsc.nsc.com (George Grenley) writes:
> So, here's the deal.  I invite Mot, Intel, and other interested parties
> to work with me in defining some sort of realistic benchmark, which we'll
> run (in public).  I expect to have system level hardware late this year,
> so if we get started now, we'll have very interesting Xmas presents...
> May the best CPU win!  

I'd like to join in the hoopla, rahrah, etc that has followed this
suggestion, and make a further one:

	Let's have the bake-off in the trade show at, say, next Winter
	Usenix.  Probably the actual setup and running of the benchmarks
	can be done a day or two before the show, so the results can be
	printed for distribution, and to give the losers time to think
	up (and print up) good explanations before we descend on them :-).

	Let's also make the same setup of machines available for people
	to run their own benchmarks.  It'd be easiest if they were all
	on a network, of course, though the benchmarks should in
	general be run from local disk to eliminate networking delays.
	Except for multi-CPU machines which want to show off their
	multitasking, only one person should be running on any machine
	at once.  But you could load a tape of benchmarks
	onto a server machine somewhere, then as time became free on
	each machine of interest, rcp over (or cp via NFS) your data,
	compile, and run.  It might even be possible to just have a
	bank of terminals where you could rlogin to each system in
	turn, using some simple scheme to avoid multiple people getting
	through at once, rather than schlepping around the trade show
	floor.  Anyone could verify the benchmark results from the
	bake-off by rerunning them themselves.

	Each such machine should have its full configuration posted
	prominently, with the list price of the configuration, if for
	sale, or its ballpark price and expected availability if the
	machine is not announced or not shipping.  If I go to Usenix
	and run some benchmarks, then go home and buy a system based on
	them, I want to be able to reproduce the configuration that won
	on my purchase order.

	Anybody who's willing to bring a machine to the trade show
	floor (and pay for the booth...) should be able to enter, e.g.
	there's no reason to restrict it to "interesting new micros".
	Of course, the benchmark machines should be available for
	benchmarking full time while the show is open, so the vendors
	should bring a second machine for demos unless they want to
	degrade their benchmark results.  To encourage prototypes to
	appear, there should be no requirement that the stuff be for
	sale yet either.  If they'll bring it, we'll benchmark it!

	Any other Usenix members interested in this?  Think we can
	get the conference committee to go for it?
-- 
Copyright 1987 John Gilmore; you may redistribute only if your recipients may.
(This is an effort to bend Stargate to work with Usenet, not against it.)
{sun,ptsfa,lll-crg,ihnp4,ucbvax}!hoptoad!gnu	       gnu@ingres.berkeley.edu