Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!wuarchive!udel!nigel.ee.udel.edu!mccalpin From: mccalpin@perelandra.cms.udel.edu (John D. McCalpin) Newsgroups: comp.arch Subject: Re: $/CPUmark is a worthless measure Message-ID: Date: 3 Apr 91 12:40:39 GMT References: <2004@kuling.UUCP> <32580014@hpcuhe.cup.hp.com> <1991Apr3.010831.3603@ico.isc.com> Sender: usenet@ee.udel.edu Organization: College of Marine Studies, U. Del. Lines: 37 Nntp-Posting-Host: perelandra.cms.udel.edu In-reply-to: rcd@ico.isc.com's message of 3 Apr 91 01:08:31 GMT >>>>> On 3 Apr 91 01:08:31 GMT, rcd@ico.isc.com (Dick Dunn) said: Dick> $/SPECmark, or $/any-CPU-benchmark, is about as useful for comparing Dick> systems as lines-of-code/day for comparing programmers, and both are Dick> about as useful as manure output for measuring the work done by horses. Dick> They're no better than order-of-magnitude, and they stink. In my posts to comp.benchmarks, I have used two MFLOPS/Million$ numbers. One used the LINPACK 1000x1000 "anything goes" results from Dongarra's report, and the other used MFLOPS based on long vector dyads which are main memory bandwidth-limited. In both cases I estimated the prices (with maximum university discounts) for a configuration of cpu 128 MB RAM (or less, if 128 MB won't fit) 1.2 GB SCSI disk no monitor, keyboard, or mouse I disagree with Dunn's comment that these ratios are no better than order-of-magnitude. I am confident that they are significant to within a factor of two, and my experience has been that they are useful for estimating both the performance on my codes and the actual cost of the system to within 25% each. The results indicate that for memory-bandwidth-limited operations, the Cray machines are approximately as cost-effective as the "Killer Micros". When system utilization is taken into account, they are likely to be significantly more cost-effective. For number-crunching, these seem to be reasonable estimates of what performance can be bought for how many $$$. Do you have any suggestions of a better approach? -- John D. McCalpin mccalpin@perelandra.cms.udel.edu Assistant Professor mccalpin@brahms.udel.edu College of Marine Studies, U. Del. J.MCCALPIN/OMNET