Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sun-barr!lll-winken!elroy.jpl.nasa.gov!usc!rpi!zaphod.mps.ohio-state.edu!magnus.acs.ohio-state.edu!cis.ohio-state.edu!pacific.mps.ohio-state.edu!linac!att!att!dptg!ulysses!andante!alice!mjh
From: mjh@alice.att.com (Mike Haertel)
Newsgroups: comp.arch
Subject: Re: Suggestions for SPEC 3.0 CPU Performance Evaluation Suite
Message-ID: <20471@alice.att.com>
Date: 27 Jun 91 17:41:53 GMT
References: <403@validgh.com> <36230004@hpindwa.cup.hp.com>
Reply-To: mjh@alice.UUCP ()
Organization: AT&T, Bell Labs
Lines: 23

In article <36230004@hpindwa.cup.hp.com> raj@hpindwa.cup.hp.com (Rick Jones) writes:
>Has anything been proposed to SPEC to measure a systems performance in
>non-standalone situations? Examples would be measures such as
>nhfsstones, and perhaps an improved ttcp (well...). The really
>interesting part would likely be DCE performance, as that seems to be
>a big step towards distributed computing. Admittedly, the problem
>becomes bigger, because the testbed is a group of systems, but isn't
>that the 'future' of computing?

Running benchmarks in a stand-alone environment is easy (or at least not
of insurmountable difficulty) for others to reproduce.

But how do you specify a standard non-standalone environment?  And how are
others supposed to reproduce it with with any degree of accuracy?  I bet
this would be an even bigger window of opportunity for manufacturers who
already do sneaky things with their benchmarks.

I think it's a very good idea though, if someone can figure out a
"foolproof" way to do it.  I just wanted to point out that it's, um,
nontrivial.

	Mike Haertel
	mjh@research.att.com