Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!usc!cs.utexas.edu!helios!auvsun1.tamu.edu!auvsaff
From: auvsaff@auvsun1.tamu.edu (David Safford)
Newsgroups: comp.benchmarks
Subject: Re: Which benchmarks are useless?
Message-ID: <15080@helios.TAMU.EDU>
Date: 22 Apr 91 14:36:56 GMT
References: <1991Apr20.083301.28886@ux1.cso.uiuc.edu> <2502@spim.mips.COM>
Sender: usenet@helios.TAMU.EDU
Reply-To: auvsaff@auvsun1.tamu.edu (David Safford)
Lines: 46

In article <2502@spim.mips.COM>, mark@mips.com (Mark G. Johnson) writes:
|>In article <1991Apr20.083301.28886@ux1.cso.uiuc.edu>
andreess@mrlaxa.mrl.uiuc.edu (Marc Andreessen) writes:
|>    >I've been reading this newsgroup since its formation.  It spends
|>    >about 95% of its time spewing out a plethora of meaningless numbers
|>    >on meaningless and trivial little pseudo-hacks which don't deserve
|>    >to be called benchmarks by any stretch of the imagination.
|>    >
|>    >These numbers are useless.
|>    >
|>
|>Useless?  Meaningless?  I would suggest that the words "useless" and/or
|>"meaningless" be reserved for benchmarks that produce results that are
|>uncorrelated (absolute value of correlation coefficient < 0.2) with
|>"correct benchmark results".
|>

--- stuff deleted

|> -- Mark Johnson	
|> 	MIPS Computer Systems, 930 E. Arques M/S 2-02, Sunnyvale, CA 94088-3650
|>	(408) 524-8308    mark@mips.com  {or ...!decwrl!mips!mark}
                                         
This is a joke, right?

You are not seriously saying that statistical correlation signifies meaning,
are you???

A statistical correlation is necessary, but certainly not sufficient, to
indicate a meaningful relationship.  Just because the per capita consumption
of M&M's correlates with the rate of bank failures does not indicate any
meaningful relationship.  This is an apple and oranges comparison.

The "date" "benchmark" is a meaningful measure of only 2 things: (1) whether
the given system uses dynamic linking, and (2) how fast date runs, given (1).

(1) is more easily determined other ways
(2) is not important to me

Let's please drop these meaningless tests, and look at more important
problems, such as network, graphics, and multiprocessing benchmark
approaches.

dave safford
auvsaff@auvsun1.tamu.edu
Texas A&M University