Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!swrinde!elroy.jpl.nasa.gov!decwrl!ogicse!sequent!muncher.sequent.com!news From: news@sequent.com (News on Muncher) Newsgroups: comp.benchmarks Subject: Re: TPC-B - is this really progress? Message-ID: <1991Apr2.224721.6234@sequent.com> Date: 2 Apr 91 22:47:21 GMT References: <1991Mar27.211436.12985@cello.hpl.hp.com> Organization: Sequent Computer Systems Inc Lines: 33 In article jonathan@cs.pitt.edu (Jonathan Eunice) writes: >Yes, but what does TPC-B measure? What useful thing, that is? If >TPC-A demands a scaled-up system that quasi-accurately reflects a >realistic OLTP use ("the actual performance you would get if you built >a local automatic teller network"), TPC-B measures what? A system >that is *not* configured like one that you'd find in a real OLTP >system? This is my impression--that TPC-B reflects a highly >synthetic, highly unrealistic configuration unlikely to be found in >real OLTP situations. Just like the TP1 rubish we've had for some >time. TP1 and TPC-B measure the performance of the database backend (the server side of the client/server model). TPC-A measures *both* the client and server sides of the problem, making it more realistic. A previous post of mine to this newsgroup explains how TPC-B is more rigorous than TP1. And in defense of the TP1, I'd have to say that this benchmark caused a lot of questionable backend database code (in terms of the ACID property) and a lot of non-performing database backend code to get cleaned up, throughout the database software industry. Sure, TPC-A is hard to do, but not impossible. Many companies have done TPC-A's; Omri Serlin's "Fault Tolerant News" newsletter's most recent issue gives a listing of all the TPC-A and TPC-B results to date. TPC-A results exist or are in progress for many platforms including HP, Sun, Unisys, Sequent, and DEC. If you really want realism, I'll bet you can hardly wait for TPC-C, the Order-Entry Benchmark. In this benchmark, the throughput is measured in Orders Per Second (yes, OPS)! This benchmark is an adaptation of one developed at DEC, and is the third benchmark in the TPC series. TPC-C's are in progress at at least one hardware vendor right now.