Path: utzoo!utgpu!news-server.csri.toronto.edu!bonnie.concordia.ca!uunet!cs.utexas.edu!bcm!convex!rosenkra
From: rosenkra@convex.com (William Rosencranz)
Newsgroups: comp.benchmarks
Subject: Re: Advice on benchmark sought
Keywords: Dhrystones
Message-ID: <1991Jan17.224000.15685@convex.com>
Date: 17 Jan 91 22:40:00 GMT
References: <1991Jan16.210617.6484@agate.berkeley.edu>
Sender: news@convex.com (news access account)
Distribution: comp
Organization: Convex Computer Corporation; Richardson, TX
Lines: 35
Nntp-Posting-Host: convex1.convex.com

In article <1991Jan16.210617.6484@agate.berkeley.edu> ilan343@violet.berkeley.edu (Geraldo Veiga) writes:
>Running dhrystone 2.1 (c version) on a 486 machine running ISC 2.2, I
>get all kinds of crazy results, with Dhrystone ratings ranging from
>3000 to 16000 in different runs.  The same machine under DOS, running
>the Drystones version in QAPLUS rates (consistently) at 15172.

no suprise here...

>Is this normal?  Is the hardware or software flaky?  If I time the run
>with Sysv's "time", user time also show the same wide variation.

dhrystone is (far) more test of your machine's compiler than the hardware
itself. i have seen similar results on systems from atari ST to cray YMP.
5:1 seems a little high; i have seen 3:1 difs between compilers on (say)
the 8 Mhz 68000 atari ST...with identical hardware. compiler opimization
switches also influence results. it sounds, however, like u see variations
with the same executable. there may be other processes
running, which is why u should always try to make bm runs on dedicated
machines, without any daemons or other processes which may steal cycles.
this is especially true with parallel systems. if ISC supports the equiv
of nice, u may try boosting the priority of the process. since MSDOS is
non-multitasking, it should give identical results (1% err) with identical
executables. however, different compilers under DOS will still give
different results, even with identical h/w.

it can be a useful bm if u are writing a compiler, or if u are shopping for
compilers, but it is not the best bm for performance analysis of h/w
itself, unless u can guarantee that each compiler will generate similar
instructions across architectures. caveat user...

-bill
rosenkra@convex.com
--
Bill Rosenkranz            |UUCP: {uunet,texsun}!convex!c1yankee!rosenkra
Convex Computer Corp.      |ARPA: rosenkra%c1yankee@convex.com