Path: utzoo!utgpu!water!watmath!clyde!att-cb!osu-cis!tut.cis.ohio-state.edu!bloom-beacon!bu-cs!bzs
From: bzs@bu-cs.BU.EDU (Barry Shein)
Newsgroups: comp.arch
Subject: Re: Tasting of Dhrystone 2.0 Results
Message-ID: <20970@bu-cs.BU.EDU>
Date: 27 Mar 88 01:52:14 GMT
References: <4076@vdsvax.steinmetz.ge.com> <3505@cbmvax.UUCP>
Organization: Boston U. Comp. Sci.
Lines: 20
In-reply-to: daveh@cbmvax.UUCP's message of 24 Mar 88 16:39:03 GMT


Wouldn't it be reasonable that, given a series of trials (perhaps by
different people) that one reports only the best result (assuming
there's no reason to believe it was due to a total error.)

I agree that this isn't how one does things in the natural sciences,
but that isn't what we're dealing with here at all. It seems that
there are a zillion reasons a benchmark might be slowed down (sudden
burst of net traffic etc) but I can't think of any good reasons that a
benchmark, properly compiled and run, would accidently run fast.
Perhaps getting lucky with a cache, but I don't think that's a concern
or is meant to be eliminated by the dhrystone methodology.

I suppose if the claim were that there was a wide variance in chip
quality (?) etc, I just don't think that's the case here.

Didn't some of the 370 series repeat and compare calculations if the
room was too warm? Now there's a wierd factor in a benchmark.

	-Barry Shein, Boston University