Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!ames!vsi1!daver!mips!mash
From: mash@mips.COM (John Mashey)
Newsgroups: comp.arch
Subject: Re: SPEC Bench-a-Thon
Message-ID: <22197@winchester.mips.COM>
Date: 24 Jun 89 17:30:26 GMT
References: <2395@canisius.UUCP> <4042@eos.UUCP>
Reply-To: mash@mips.COM (John Mashey)
Organization: MIPS Computer Systems, Inc.
Lines: 42

In article <4042@eos.UUCP> jaw@eos.UUCP (James A. Woods) writes:
>>>as a benchmark, what's wrong with unix 'compress',
>>>other than it messes up mipsco internal benchmark numbers?
>>>
>>  We used compress as one of many tests between a MIPS system and several
>>  others.  It didn't "mess up" the MIPS in any way.  
>> 
>was referring here to the bit about how this integer benchmark,
>since it scribbles semi-randomly over a sparse array bigger
>than the cache, runs at about nine-ten vax mips on the m3000 system,
>which is billed at twenty.
>
>a mipsco employee informed me that the problematic memory write latency
>exercised by 'compress' "dragged down their numbers for the r3000 chip"
>during simulations.
>
>other risc implementations, of course, may yield similar results.

1) MIPS uses compress as one of its internal benchmarks.
2) It does indeed drag down the average; that's why we included it,
because of its nasty effect on caches.
3) It's on the "proposed" list for SPEC [by me], but I just ran out
of time to get it packaged, and certain characteristics now make
it look a little more appropriate for evaluation in the second round.
These characteristics are:
	the first round of the SPEC benchmarks are essentially CPU-bound
	programs, that is, on any machine big enough not to have
	them cause noticable paging, the elapsed time approx= user-cpu+
	system-cpu, which means it's easy to agree on how to measure
	and report it. (i.e., elapsed time).

compress, to be used for SPEC, had to be made to run much longer than
the case MIPS uses internally.  In doing so, it now turns into a benchmark
that: a) either does serious I/O, or b) has elapsed time that might be
substantially longer than CPU time.   We're going to be working on
a coherent policy of handling such things, but we'd like to get
the first tape out the door first.
-- 
-john mashey	DISCLAIMER: <generic disclaimer, I speak for me only, etc>
UUCP: 	{ames,decwrl,prls,pyramid}!mips!mash  OR  mash@mips.com
DDD:  	408-991-0253 or 408-720-1700, x253
USPS: 	MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086