Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!lll-winken!lll-lcc!pyramid!prls!mips!hansen
From: hansen@mips.UUCP (Craig Hansen)
Newsgroups: comp.arch
Subject: Re: Towards A Meaningful Performance Measure
Message-ID: <866@mips.UUCP>
Date: Mon, 2-Nov-87 18:29:12 EST
Article-I.D.: mips.866
Posted: Mon Nov  2 18:29:12 1987
Date-Received: Fri, 6-Nov-87 03:58:33 EST
References: <861@winchester.UUCP> <2993@phri.UUCP> <864@tut.cis.ohio-state.edu> <3806@sol.ARPA>
Lines: 35
Keywords: benchmarks

In article <3806@sol.ARPA>, crowl@cs.rochester.edu (Lawrence Crowl) writes:
> The compiler and operating system also affect performance.  To make the base
> machine highly available, both should be common.  I suggest Unix BSD 4.2 as
> the base operating system and the portable C compiler as the base C language
> compiler.

BSD 4.3 has been out long enough to be "common," and has significant
improvements in the performance of compiled Fortran code, and some (more
modest) improvements in C code. VMS has better compilers, both C and Fortran,
in terms of how the resulting code perform. The availability of a common
machine is important, however. MIPS has both 4.3 and VMS 780's, but no longer
keeps a 4.2 around. It's been obsoleted by 4.3.

> Of course, 780's are becoming scarce.  We may have to pick another machine just
> to keep the base machine readily available.  Suggestions?

Digital Review uses the roughly equivalent MicroVax as a base machine in their
comparisons.

Ultimately, the Vax Relative Performance measure will never be much more
precise than defining the inch as "Three round and dry barleycorns laid end to
end." In fact, it's more like a cubit - it depends on what the leading machine
of the day is (or was). The 780 VRP at best provides rough comparisons of
machine performance, and by all rights ought to be a moving target. Since we
(at MIPS) are clearly making an effort to be using the best compiler system we
can muster when benchmarking our machines, we consider it only fair to use the
best compiler system that can be made available for the VAX 780. We are trying
to be conservative as possible in our performance claims in an environment
where inflated and misleading claims are the order of the day, but at times it
seems to work against us....

-- 
Craig Hansen
Manager, Architecture Development
MIPS Computer Systems, Inc.
...decwrl!mips!hansen