Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!husc6!bbn!rochester!udel!princeton!rutgers!dayton!umn-cs!hall!pmk
From: pmk@hall.cray.com (Peter Klausler)
Newsgroups: comp.arch
Subject: Re: Benchmarking in a High Level Language
Message-ID: <1444@hall.cray.com>
Date: Wed, 4-Nov-87 14:37:02 EST
Article-I.D.: hall.1444
Posted: Wed Nov  4 14:37:02 1987
Date-Received: Mon, 9-Nov-87 04:19:47 EST
References: <861@winchester.UUCP> <2993@phri.UUCP> <864@tut.cis.ohio-state.edu> <18966@amdcad.AMD.COM>
Organization: Cray Research, Inc., Mendota Heights, MN
Lines: 39
Keywords: benchmarks
Summary: X-MP caveat

In article <18966@amdcad.AMD.COM>, tim@amdcad.AMD.COM (Tim Olson) writes:
| In article <3285@ames.arpa> eugene@pioneer.UUCP (Eugene Miya N.) writes:
| | In article <3806@sol.ARPA> crowl@cs.rochester.edu (Lawrence Crowl) writes:
| | >Of course, 780's are becoming scarce.  We may have to pick another machine just
| | >to keep the base machine readily available.  Suggestions?
| | Yes, the Cray X-MP.
| Another problem: it is not a very common computer.  Nearly everyone who
| is doing some serious benchmarking at least has access to a VAX.

Beware of using "X-MP" as a GENERIC unit of performance.
Even for a single-processor benchmark using only instructions common
to all X-MPs, significant performance differences arise between
different machines due to:

	1) Different clock rates. The clock period on an X-MP is
	   adjustable (within a fairly narrow range),
	   and current X-MPs are configured with a
	   clock period 1.0ns faster than original 1983 vintage X-MPs.
	2) Different memory parts (MOS or bipolar).
	3) Different memory bank configurations.

If your benchmark code can be multiprocessed, or use the special
instructions found only on some X-MPs, then you'll get even more
variation between machines.

As with any other computer available is more than one configuration,
please provide complete information about your X-MP when publishing
benchmark results. Statements like "Fortran on a Cray X-MP: 34.3 sec"
just aren't sufficient; please specify whether you're using an X-MP/14se
or a full-blown X-MP/416.

(After reviewing this before posting, it occurs to me that most comp.arch
 readers are probably aware of what I've brought up. Forgive me if I'm being
 redundant.)

      Peter Klausler			       pmk@hall.cray.com
   Compiler Development			...!{ihnp4,sun!tundra}!hall!pmk
    Cray Research, Inc.			        (612) 681-5828
Mendota Heights, Minnesota	I like languages with P's in their names.