Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!usc!sdd.hp.com!hplabs!hpfcso!jvm
From: jvm@hpfcso.FC.HP.COM (Jack McClurg)
Newsgroups: comp.benchmarks
Subject: Re: Running the SPEC benchmarks (pedantic questions)?
Message-ID: <21720005@hpfcso.FC.HP.COM>
Date: 7 Apr 91 05:58:01 GMT
References: <2767@sirius.ucs.adelaide.edu.au>
Organization: Hewlett-Packard, Fort Collins, CO, USA
Lines: 17

Without answering the questions asked here ( I think that you have an excellent
series of questions; I hesitate to answer, since my answer may be thought to
be "official" and it is not; Some reported results probably have used some of
the techniques described; BTW, which do you think are legitimate?), I merely
point out that SPEC distributes the source code to the SPEC benchmarks so that
the results printed in the SPEC newsletter may be reproduced.  Also, the
results summary page should have complete configuration information and notes
sufficient to ensure reproducibility.

If you have difficulty duplicating results, call the vendor for assistance,
and then tell us what transpired.  This should include instances where
your results were better than what the vendor claimed.

Jack McClurg
jack_mcclurg@fc.hp.com
303-229-2126
Chair, SPEC Steering Committee