Path: utzoo!attcan!uunet!seismo!sundc!pitstop!sun!decwrl!nsc!grenley
From: grenley@nsc.nsc.com (George Grenley)
Newsgroups: comp.arch
Subject: Re: Bar Wars again (benchmarking)
Message-ID: <6729@nsc.nsc.com>
Date: 1 Oct 88 04:22:18 GMT
References: <2220003@hpausla.HP.COM> <46500026@uxe.cso.uiuc.edu> <6683@nsc.nsc.com> <6684@nsc.nsc.com> <4263@wright.mips.COM>
Reply-To: grenley@nsc.UUCP (George Grenley)
Organization: National Semiconductor, Sunnyvale
Lines: 63

The response this time seems less enthusiastic than before...

In article <4263@wright.mips.COM> earl@mips.COM (Earl Killian) writes:
>In article <6684@nsc.nsc.com> grenley@nsc.nsc.com (George Grenley) writes:
>   Many moons ago I proposed
>   on this net a public, head to head comparison of the various 32 bit
>   chips 
>   Each prospective participant should bring his iron, and the usual
>   benchmark code.  Be prepared to have ALL code inspected (including
>   library sources) by the referees.  We will compile and run the
>   code, and all results will be published, and placed in the public
>   domain.  We will also verify clock rates, etc.
>
>The logistics of this are a problem.  Why don't you produce a
>performance brief along the lines of the MIPS Performance Brief and
>then post it, as MIPS has done.

At last, someone at NSC is working on a comprenhensive benchmark summary.
Better late than never.  For various reasons, we at NSC have not been 
agressive in producing scientifically controlled benchmark data.

I do want to complement MIPS for the quality of their reports; in fact, I
have used them internally as examples of how it ought to be done.

I had several reasons for my proposal, though.  One was to provide an unofficial
engineer-only environment.  AT amny companies, pressures exist to go with
some official number, rather than measured data.  Also, there are frequently
variations in supposedly standard code.  I have different versions of Dhry1.1
which vary over 40%, even though they are supposedly the same code.

Also, the matter of verification remains.  LAst February, I saw a Mot 030
machine (at Buscon) running in excess of 13,000 Dhrystones.  They would not
show me the source, or let me touch it in any way.  I can only wonder about
that number - no other 030 user I know of has been able to verify it.

By doing a demo publicly, all participants can examine one anothers' machines
and code in whatever detail seems necessary.

Logistics may be awkward for some, but most participants are right here in 
Silicon Valley.

>To start with, what are your results on the following public-domain
>benchmarks?
>
>livermore fortran kernels, harmonic, geometric,
>	and arithmetic mean, sp and dp
>
>linpack 100x100, fortran and coded blas, sp and dp
>
>1000x1000, sp and dp
>
>spice2g6, bipole, digsr, and comparator inputs (as posted on comp.arch
>	last year)
>
>Hmm, as usual, no decent public-domain integer benchmarks come to
>mind.  MIPS uses diff, grep, yacc, nroff, espresso, and timberwolf as
>integer benchmarks given suitable inputs, but these aren't
>public-domain, and aren't generally agreed upon.

I will post this info as soon as I have it, but it will be awhile - I am just
getting started, and I don't even have sources for some of this yet.

George Grenley