Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!mips!mash
From: mash@mips.com (John Mashey)
Newsgroups: comp.benchmarks
Subject: Re: Price/Performance figures for Number-Crunching
Message-ID: <1628@spim.mips.COM>
Date: 29 Mar 91 20:21:44 GMT
References: <MCCALPIN.91Mar20163118@pereland.cms.udel.edu> <1991Mar21.000302.10103@convex.com> <1991Mar23.192405.7668@aeras.uucp>
Sender: news@mips.COM
Organization: MIPS Computer Systems, Inc.
Lines: 28
Nntp-Posting-Host: winchester.mips.com

In article <1991Mar23.192405.7668@aeras.uucp> jt@aeras.UUCP (J T McDuffie) writes:
>>
>        If you receive the SPEC newsletter you'll find that they have
>        started to utilize multiple figures of merit to describe any
>        given machine, feeling that THERE IS NO SINGLE FIGURE OF MERIT.
>        Thus SPEC has the SPECmark, SPECthruput, SPECint (SPECmark
>        considering INTEGER only CPU benchmark components), and SPECfp
>        (SPECmark considering FLOATING POINT only CPU benchmark
>        components).
>
>	So it would seem that at least SPEC would agree with McGehearty
>	that you need to evaluate a NUMBER of benchmark results when
>	attempting to characterize any given system/architecture.
SPEC has ALWAYS believed that there is no one figure of merit.
That's why SPEC has always used a reporting form that included all 10 numbers,
and why we've always said you need to see all of the numbers to even
get a clue what the machine is like.  It turns out that the SPECint
metric (the 4 integer C benchmarks) is relatively stable, i.e.,
it has a fairly narrow confidence interval, small variance, etc,
on every machine ever measured.  SPECfp tends to vary a lot more,
i.e., it is alwasy good to run your own benchmarks, but you'd be especially
tempted to do so when comparing machines on FP performance, because it
varies all over the map, especially on the more vectorizable ones.
-- 
-john mashey	DISCLAIMER: <generic disclaimer, I speak for me only, etc>
UUCP: 	 mash@mips.com OR {ames,decwrl,prls,pyramid}!mips!mash 
DDD:  	408-524-7015, 524-8253 or (main number) 408-720-1700
USPS: 	MIPS Computer Systems MS 1/05, 930 E. Arques, Sunnyvale, CA 94086