Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!seismo!sundc!pitstop!sun!decwrl!decvax!necntc!husc6!bbn!rochester!crowl
From: crowl@cs.rochester.edu (Lawrence Crowl)
Newsgroups: comp.arch
Subject: Re: Towards A Meaningful Performance Measure
Message-ID: <3907@sol.ARPA>
Date: Wed, 4-Nov-87 13:40:27 EST
Article-I.D.: sol.3907
Posted: Wed Nov  4 13:40:27 1987
Date-Received: Sat, 7-Nov-87 16:29:15 EST
References: <861@winchester.UUCP> <2993@phri.UUCP> <864@tut.cis.ohio-state.edu> <3806@sol.ARPA> <1714@pdn.UUCP>
Reply-To: crowl@cs.rochester.edu (Lawrence Crowl)
Organization: U of Rochester, CS Dept, Rochester, NY
Lines: 25
Keywords: benchmarks

In article <1714@pdn.UUCP> alan@pdn.UUCP (0000-Alan Lovejoy) writes:
>I strongly vote against PCC as the compiler.  It is known to vary considerably
>in code-generation quality on different architectures.  Not good.  It's better
>if people use the best compiler avaiable for the system, whatever that is.

My intent was that the base architecture/operating system/compiler be constant.
This means that some readily available compiler be part of the base system.  I
wanted to avoid cases where people benchmark relative to two different
compilers.  I want people to use the compiler appropriate to their machine.

>As for operating system, I suggest an average based on standalone, UNIX V and
>the most widely used OS on that system, if that is not UNIX V.  UNIX V is
>clearly headed for standard status.  Leave UNIX out if it's not available.  I
>also strongly urge against UNIX utilities as benchmarks:  not everyone uses or
>can use UNIX (a sacrilege, I know).

I do not understand your reasoning for averaging UNIX and something else.  Why
not just provide two number, one for each operating system on that machine.  I
suspect that the OS will have little effect on CPU bound jobs (relative to the
compiler anyway).  The reason for choosing UNIX utilities is that those people
interested in a UNIX box have more realistic measures of performance.
-- 
  Lawrence Crowl		716-275-9499	University of Rochester
		      crowl@cs.rochester.edu	Computer Science Department
...!{allegra,decvax,rutgers}!rochester!crowl	Rochester, New York,  14627