Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!usc!cs.utexas.edu!helios!auvsun1.tamu.edu!auvsaff
From: auvsaff@auvsun1.tamu.edu (David Safford)
Newsgroups: comp.benchmarks
Subject: Re: Which benchmarks are useless?
Keywords: benchmarks  date  statistical correlation
Message-ID: <15159@helios.TAMU.EDU>
Date: 23 Apr 91 14:07:48 GMT
References: <18049@sunquest.UUCP>
Sender: usenet@helios.TAMU.EDU
Reply-To: auvsaff@auvsun1.tamu.edu (David Safford)
Distribution: usa
Lines: 47

In article <18049@sunquest.UUCP>, terry@venus.sunquest.com (Terry R.
Friedrichsen) writes:
|>auvsaff@auvsun1.tamu.edu (David Safford) writes:
|>>mark@mips.com (Mark G. Johnson) writes:
|>>
|>>|>I would suggest that the words "useless" and/or
|>>|>"meaningless" be reserved for benchmarks that produce results that are
|>>|>uncorrelated (absolute value of correlation coefficient < 0.2) with
|>>|>"correct benchmark results".
|>>
|>>You are not seriously saying that statistical correlation signifies meaning,
|>>are you???
|>>
|>>A statistical correlation is necessary, but certainly not sufficient, to
|>>indicate a meaningful relationship.  Just because the per capita consumption
|>>of M&M's correlates with the rate of bank failures does not indicate any
|>>meaningful relationship.  This is an apple and oranges comparison.
|>
|>No, and he didn't say that.  Read what he wrote:  "useless == uncorrelated".
|>He did NOT say "correlated == useful".  You yourself admit that correlation
|>is a necessary, though insufficient, condition.
|>

YOU read it again, slowly and carefully.  He said that "useless" should
be RESERVED for uncorrelated benchmarks.  This means that if a benchmark
does correlate, it cannot be called useless, as this is RESERVED for 
uncorrelated.  Using your nomenclature, he said "correlated != useless".
Using the reasonable assumption that the sets of "useful" and "useless"
benchmarks are disjoint and complementary, we can say that 
"!= useless" == "useful", and thus his statement reduces to
"correlated == useful".  To support the assumption that "useful" and
"useless" are disjoint and complementary, attempt a contradiction.
Suppose that the sets are not disjoint.  This means that there must
exist at least on benchmark that is both "useful" and "useless", clearly
a contradiction.  Suppose that there exists a benchmark that is neither
"useful", nor "useless".  But by definition, since this benchmark is not
"useful", it must be "useless".  Thus the reduction "!= useless" == "useful"
must hold. QED.
so he DID say, using your nomenclature, "correlated == useful".

|>One of the real problems I note in reading this group is that there are a
|>LOT of folks who are weak on logic, in interpretation of benchmarks as well
|>as other postings.
|>

You provided an excellent example!                                      
Beam me up, Scotty, there's no intelligent life in THIS newsgroup.