Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!utgpu!water!watmath!clyde!rutgers!mit-eddie!think!ames!ucbcad!ucbvax!decvax!decwrl!sun!wdl1!bobw
From: bobw@wdl1.UUCP
Newsgroups: comp.arch
Subject: Re: Benchmarking ...[really Nelson benchmarks]
Message-ID: <3490003@wdl1.UUCP>
Date: Tue, 26-May-87 13:25:58 EDT
Article-I.D.: wdl1.3490003
Posted: Tue May 26 13:25:58 1987
Date-Received: Wed, 27-May-87 06:45:46 EDT
References: <410@winchester.UUCP>
Lines: 29

/ wdl1:comp.arch / mash@mips.UUCP (John Mashey) /  1:36 am  May 21, 1987 /
In article <6024@steinmetz.steinmetz.UUCP> davidsen@kbsvax.steinmetz.UUCP (William E. Davidsen Jr) writes:
>
>As the developer of a benchmark suite of my own I would love to cast
>bricks at the Nelson suite. In truth it's a pretty good set of benchmarks,
>and has been run on hundreds of configurations. I agree that it would be
>a suitable measure of machines.

One "curious" point about the Nelson benchmarks is their intentional
use of goto throughout the c source. This is supposed to make the
results independent of optimizing in the compilers on different
systems.

I've never been quite sure what that accomplishes. To put it another
way, what is the benchmark supposed to be measuring: SYSTEM
performance, or HARDWARE performance. If the first, then a
sophisticate compiler/optimizer seems to me a significant part of
systerm performance. After all, the application code you run on the
system will presumably get some benefit out of the optimizer too.
If the second, what relevance has it to buying a SYSTEM? What the
customer presumably cares about is how well the system runs his/her
application, not the rate at which it can access memory or increment
a register. (The latter may well be of interest to system designers,
looking for bottlenecks to relieve, or for marketing types trying to
design a benchmark that makes their system look better than the
competition, but Nelson is being used to make purchase decisions.)

-----------------------------------------------------------------
I disclaim almost everything, probably including this line.