Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!mips!mash From: mash@mips.com (John Mashey) Newsgroups: comp.benchmarks Subject: Re: Which benchmarks are useless? Keywords: benchmarks date statistical correlation Message-ID: <3001@spim.mips.COM> Date: 3 May 91 16:02:30 GMT References: <1751@marlin.NOSC.MIL> <2800@spim.mips.COM> <1756@marlin.NOSC.MIL> Sender: news@mips.COM Distribution: comp.benchmarks Organization: MIPS Computer Systems, Inc. Lines: 75 Nntp-Posting-Host: winchester.mips.com In article <1756@marlin.NOSC.MIL> aburto@marlin.NOSC.MIL (Alfred A. Aburto) writes: >>As a minor point, for whatever reason, most of the dhrystone-vax-mips >>ratings in the world assume VAX-11/780 = 1,757 1.1 Dhrystones, >>which slightly raises the numbers everywhere. >Yes, I was aware of that, but I felt constrained to use the peak numbers >as given in that article and the article indicated 1870 Dhrys/sec (1.1) >peak for the the VAX 11/780. I've seen the 1757 Dhrys/sec (V1.1) >referenced in IBM advertisements for their POWERstations, but that is all >I know about that number. That's the problem, of course with using base numbers that can change around on you [which is why the SPEC base numbers are frozen for eternity]. Using 1757 as the base are {Sun, HP, IBM, Motorola, and many others} So, I'll list here some various combinations, expressed both as SPECint, as in vax-mips-using-dhrystone-1.1, assuming 1757 is the base (because all of the following assume that, I think): SPECint dhry-mips Ratio System Notes 16.4 47.8 2.7X i860 @ 33Mhz SPECint published for Intel Star860, Summer 90; Dhrys from March 1989 i860 Performance Brief 15.8 27.5 1.7 RS/6000 320 SPEC #s and ads 12.9 20+ 1.6 HP 9000/425s 68040@ 25Mhz (I may be wrong on the 20, although that's what Moto usually calls a 25Mhz 68040, and the HP published Dhyr-mips number might actually be higher, but I can't seem to find it handy.) 26.4 41.5 1.6 MIPS Magnum/33 33MHz R3000A (NOTE: just to be perfectly clear, MIPSco has NEVER used a Dhrystone-mips rating as "the mips-rating" for a system. On our internal scale (our own set of benchmarks) this rates at 27, i.e., very close to SPECint). 38.1 57 1.5 HP Snakes,50Mhz [57 is from all of the ads...] 17.7 27.0 1.5 i486 @ 33Mhz Intel 486 Performance Brief, 1Q91. 20.7 28.5 1.4 Sun SS2 40Mhz Cypress SPARC >Based on the results I'd say Dhry(A) and Dhry(B) yield VAX-MIPS ratings >that are 14% to 24% high WHEN COMPARED to SPECint(A) and SPECint(B) VAX-MIPS >ratings. I'd hesitate to infer anything beyond that as I'm still seeking >more information. Well, above we have: 1 each of {IBM, MIPS, HP PA, SPARC} 1 each of {i486, 68040} which, I think covers a fair chunk of current computers..... And the LOWEST expansion was 40%..... Now, Dhrystone 2.1 generally would deflate these numbers by approx 10%; getting rid of strcpy-inlining would deflate by at another 20-30%, more in some cases. [For example, over the last few years, the best way to boost your Dhrystone was to put in strcpy/strcmp inlining, even though that has seldom been found to help realistic programs very much. the fiercest example of this is for the i860, whose code does the following: starts with strcpy of 30-byte constant string pads the string and the target to 32-bytes, which happens to be possible in this case aligns the string on 8-byte or better boundaries inlines the strcpy generates 2 16-byte loads and 2 16-byte store, which, being to and from a writeback cache, with zero-cache misses, goes fast. Of course, this bears no resemblance to any realistic character-pushing, and that fact rapidly shows up when you see the SPEC integer numbers...] Fortunately, (except in some ads in the Wall Street Journal :-), companies are emphasizing SPEC more and more these days. -- -john mashey DISCLAIMER: UUCP: mash@mips.com OR {ames,decwrl,prls,pyramid}!mips!mash DDD: 408-524-7015, 524-8253 or (main number) 408-720-1700 USPS: MIPS Computer Systems MS 1/05, 930 E. Arques, Sunnyvale, CA 94088-3650