Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sdd.hp.com!zaphod.mps.ohio-state.edu!unix.cis.pitt.edu!pitt!vax.cs.pitt.edu!jonathan From: jonathan@cs.pitt.edu (Jonathan Eunice) Newsgroups: comp.benchmarks Subject: Re: TPC-B - is this really progress? Message-ID: Date: 2 Apr 91 02:46:55 GMT References: <1991Mar27.211436.12985@cello.hpl.hp.com> <31798@shamash.cdc.com> Sender: news@pitt.UUCP Organization: University of Pittsburgh Computer Science Lines: 19 In-reply-to: jeff@u02.svl.cdc.com's message of 28 Mar 91 03:02:54 GMT renglish@cello.hpl.hp.com (Bob English) writes: >In addition, TPC-A numbers are difficult to get. It takes many runs, >using lots of equipment, to get an accurate assessment. For systems not >intended for the marketplace described above, this level of effort is >difficult to justify. Many replys have mentioned the cost/difficulty of getting these numbers. Fair enough. This seems to indicate that TPC did not do a terribly good job defining benchmarks that are both useful and doable. But what about my original premise--that if all we have are TPC-B/TP1 numbers, we basically have no terribly valuable quantification of what a system is capable of? That we have numbers no more useful, and perhaps less useful, than the MIPS/MFLOPS figures so abused in the technical computing arena? If all TPC has does is to give the weight of "industry standard" and a patina of respectability to empty metrics, ought we call this progress?