Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!unix.cis.pitt.edu!pitt!vax.cs.pitt.edu!jonathan From: jonathan@cs.pitt.edu (Jonathan Eunice) Newsgroups: comp.benchmarks Subject: TPC-B - is this really progress? Message-ID: Date: 25 Mar 91 22:51:31 GMT Sender: news@pitt.UUCP Organization: University of Pittsburgh Computer Science Lines: 17 I notice a number of vendors, such as Sun and (given a recent comp.arch posting) DG, concentrating on TPC-B benchmarks, rather than TPC-A. With all due respect to the TPC folks, who do appear to be trying to make the world safer for benchmarking, isn't this the same deal we had before with TP1 (the TPC-B precursor) benchmarks? The deal whereby vendors get to quote absurdly high #s that don't reflect real life? Oughtn't we be concentrating on the more-fully-scaled TPC-A runs, and stongly encouraging our vendors to do likewise? Also, I notice that Sun used two machines to do their TPC-B runs -- either a SS2 and a SS1+ (then quoting this in their spec sheets as the SS2 figure), or a SS490 and a SS2 (result quoted as SS490 value). I don't know the precise TPC rules, but it seems a little un-kosher to me. (Kind of like Motorola when it slipped through the first draft of the SPEC reporting rules, forcing the quick adoption of SPECthru numbers.) Am I wrong? Is it legit? Are other vendors doing it? (I know HP and IBM haven't.)