Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!sdd.hp.com!hplabs!hpfcso!maf
From: maf@hpfcso.FC.HP.COM (Mark Forsyth)
Newsgroups: comp.benchmarks
Subject: Re: Price/Performance figures for Number-Crunching
Message-ID: <21720002@hpfcso.FC.HP.COM>
Date: 31 Mar 91 22:02:36 GMT
References: <MCCALPIN.91Mar18165912@pereland.cms.udel.edu>
Organization: Hewlett-Packard, Fort Collins, CO, USA
Lines: 37


<From: mash@mips.com (John Mashey)
<
<It turns out that the SPECint
<metric (the 4 integer C benchmarks) is relatively stable, i.e.,
<it has a fairly narrow confidence interval, small variance, etc,
<on every machine ever measured.  SPECfp tends to vary a lot more,
<i.e., it is alwasy good to run your own benchmarks, but you'd be especially
<tempted to do so when comparing machines on FP performance, because it
<varies all over the map, especially on the more vectorizable ones.

You seem to be confusing "performance relative to a VAX 11/780" with
"performance".  Since the units chosen for the SPEC numbers are relative
to a VAX 11/780, a flat curve on the individual components means only
that the system faithfully reproduces all of the relative strengths, and
weaknesses, of the VAX, NOT that it is necessarily a "balanced" design. 
A small variance is not necessarily a good goal to design for.  The 780 
was very good at integer intensive workloads, but, one would expect that 
modern RISCs with 64 bit floating point units and an additional 10 years 
of evolution in compiler technology to perform relatively better on FP 
intensive applications. The FP components of the SPEC suite also seem 
to do a better job at measuring memory hierarchy performance - we see the 
highest cache and TLB miss rates on these. There is a lot of variation from 
machine to machine and benchmark to benchmark on the floating point intensive 
components of the SPEC suite but, presumably all of these were considered 
to be representations of important classes of customer applications on 
workstations. I agree that the only good measure of a machine's performance 
is on the actual applications that each user will run, especially for 
users who care as much about graphics or I/O performance as they do about
integer and FP computation intensive applications. 


disclaimer: these are my opinions, not HP's 

Mark Forsyth
Hewlett Packard Company
Fort Collins, Colorado