Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!utgpu!water!watmath!clyde!rutgers!mit-eddie!genrad!decvax!decwrl!pyramid!prls!mips!sjc
From: sjc@mips.UUCP
Newsgroups: comp.arch,comp.sys.nsc.32k
Subject: Re: "Unoptimizing" Dhrystone
Message-ID: <999@mips.UUCP>
Date: Wed, 15-Apr-87 22:44:13 EST
Article-I.D.: mips.999
Posted: Wed Apr 15 22:44:13 1987
Date-Received: Fri, 17-Apr-87 05:36:20 EST
References: <4190@nsc.nsc.com> <951@moscom.UUCP> <2577@intelca.UUCP> <864@oakhill.UUCP>
Lines: 37
Xref: utgpu comp.arch:889 comp.sys.nsc.32k:71

In article <864@oakhill.UUCP>, davet@oakhill.UUCP (Dave Trissel) writes:
> In article <16151@amdcad.AMD.COM> tim@amdcad.UUCP (Tim Olson) writes:
> >In article <219@homxb.UUCP> gemini@homxb.UUCP (Rick Richardson) writes:
> >>                               ...  Meanwhile, any advice on modifying
> >>the Dhrystone for version 1.2 such that a global optimizer won't be
> >>able to remove anything will be appreciated.
> >
> >One easy fix to Dhrystone is to package it as two (or more) separate c
> >files which must be compiled separately, then linked together...
> 
> Note that there is at least one company, Philon, Inc. which claims to have
> an optimization pass which runs on the final linked object code which would
> defeat the above tactics.

In any benchmark, if you compute some function (e.g. a hash) of all of
the variables which are set but not otherwise used, and then print the
result, you not only prevent optimizers from eliminating the code which
sets those variables, but you also provide a check on whether the
emitted program is correct. It's human nature to concentrate on how
fast a benchmark runs and ignore whether it runs correctly, but the
latter is really a prerequisite, particularly when you're working with
a compiler or optimizer which is in a state of flux.

My plea to benchmark designers:  measure whatever time you want, but
afterward compare the results against known correct constant values,
and print a go/no-go message telling whether the results were correct
(or, for floating point, whether they were within an acceptable range).

Dhrystone has additional vulnerable spots: for example, the empty loops
used for subtracting out the loop-overhead. But the loops don't have to
be empty; you could add some nontrivial computation (whose result is
later printed, compared against a constant, or used to compute a hash)
to both the empty loops and the real ones, so that the subtraction
removes both the effect of the looping and the effect of the computation.

-- 
...decwrl!mips!sjc						Steve Correll