Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!pacific.mps.ohio-state.edu!linac!att!news.cs.indiana.edu!ux1.cso.uiuc.edu!mrlaxs.mrl.uiuc.edu!andreess
From: andreess@mrlaxs.mrl.uiuc.edu (Marc Andreessen)
Newsgroups: comp.benchmarks
Subject: Re: Which benchmarks are useless?
Message-ID: <1991Apr21.001124.14058@ux1.cso.uiuc.edu>
Date: 21 Apr 91 00:11:24 GMT
References: <1991Apr20.083301.28886@ux1.cso.uiuc.edu> <2502@spim.mips.COM>
Sender: usenet@ux1.cso.uiuc.edu (News)
Organization: University of Illinois, Urbana
Lines: 36

In article <2502@spim.mips.COM> mark@mips.com (Mark G. Johnson) writes:
>Since Andreessen is at the University of Illinois, let's pretend,
>temporarily, that the Illinois Perfect Club is the definition of a
>correct benchmark.  Then a useless benchmark is one which, after being
>run on a large subset of the same machines as have run the Illinois
>Perfect Club, produces a correlation coefficient r in the range
>(-0.2 < r < 0.2)  -- that is, the candidate benchmark's results are
>uncorrelated with the Illinois Perfect Club results.

You're joking.  I hope.  The Perfect Club (which I have nothing to
do with; my address does not include 'csrd.uiuc.edu') benchmarks only
one thing:  execution time of a given subset of commonly-used
scientific applications.  The hope is that by examining the Perfect Club
numbers, one might obtain an estimation of how fast one's own
similarly-coded application would execute on a given machine.

This says nothing about a broad range of interesting and not-so-interesting
facets of computation (aka dates per second).

If you really want to know how many dates per second your machine
can generate, that's fine, but the numbers are still useless. 
I'm a little bit more concerned with how fast I can execute a given 
scientific application than how many dates I can spew out.  Curiously 
enough, those concerns are shared with most of the scientific community; 
thus, Linpack, the Livermore Loops, and Perfect Club.

Having said that, I might further humbly submit that dates/second is also
a useless measure of OS performance.  This should be obvious; if not,
go back and read the original posting, wherein the author lists a wide
range of reasons this hack is useless as a benchmark.

Marc

-- 
Marc Andreessen___________University of Illinois Materials Research Laboratory
Internet: andreessen@uimrl7.mrl.uiuc.edu____________Bitnet: andreessen@uiucmrl