Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!rice!uw-beaver!zephyr.ens.tek.com!tektronix!sequent!sweiger From: sweiger@sequent.UUCP (Mark Sweiger) Newsgroups: comp.benchmarks Subject: Re: TPC-B - is this really progress? Message-ID: <56476@sequent.UUCP> Date: 29 Mar 91 19:38:50 GMT References: Reply-To: sweiger@eng2.UUCP (Mark Sweiger) Organization: Sequent Computer Systems, Inc. Lines: 44 In article jonathan@cs.pitt.edu (Jonathan Eunice) writes: >I notice a number of vendors, such as Sun and (given a recent >comp.arch posting) DG, concentrating on TPC-B benchmarks, rather than >TPC-A. With all due respect to the TPC folks, who do appear to be >trying to make the world safer for benchmarking, isn't this the same >deal we had before with TP1 (the TPC-B precursor) benchmarks? The >deal whereby vendors get to quote absurdly high #s that don't reflect >real life? Oughtn't we be concentrating on the more-fully-scaled >TPC-A runs, and stongly encouraging our vendors to do likewise? It is very hard to get high rates of throughput in a TPC-A test. It is even harder to get good $/TPS in a TPC-A test. So there is a tendency to retreat to easier ground, the TPC-B. However, TPC-B is a better test than the TP1 in that it requires no single point of failure (implies mirrored DBMS log), requires log archiving while the test is running, and requires at least one checkpoint during the test run. It also has some strict full disclosure rules. > >Also, I notice that Sun used two machines to do their TPC-B runs -- >either a SS2 and a SS1+ (then quoting this in their spec sheets as the >SS2 figure), or a SS490 and a SS2 (result quoted as SS490 value). I >don't know the precise TPC rules, but it seems a little un-kosher to >me. (Kind of like Motorola when it slipped through the first draft of >the SPEC reporting rules, forcing the quick adoption of SPECthru >numbers.) Am I wrong? Is it legit? Are other vendors doing it? (I >know HP and IBM haven't.) It doesn't matter how many machines you use. The Sun systems put clients (tellers) on one machine, the RDBMS server or servers on the other machine. You cannot have more than one server machine unless the software allows a database to be distributed in a consistent (ACID property) fashion across nodes. The notable example of this is the recent Oracle 6.2/VAX cluster test which reported 425 TPC-B TPS with a 4 node cluster. In general, the fewer machines you use for the test, the better your $/TPS. -- Mark Sweiger Sequent Computer Systems Database Software Engineer 15450 SW Koll Parkway Office: (503)578-4329 Beaverton, Oregon 97006-6063 FAX: (503)578-7569 sweiger@sequent.com