Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!sdd.hp.com!hplabs!cello!renglish From: renglish@cello.hpl.hp.com (Bob English) Newsgroups: comp.benchmarks Subject: Re: TPC-B - is this really progress? Message-ID: <1991Apr03.172801.15802@cello.hpl.hp.com> Date: 3 Apr 91 17:28:01 GMT References: Organization: Hewlett Packard Labs Lines: 32 jonathan@cs.pitt.edu (Jonathan Eunice) writes: > But what about my original premise--that if all we have are TPC-B/TP1 > numbers, we basically have no terribly valuable quantification of what > a system is capable of? That we have numbers no more useful, and perhaps > less useful, than the MIPS/MFLOPS figures so abused in the technical > computing arena? If all TPC has does is to give the weight of "industry > standard" and a patina of respectability to empty metrics, ought we call > this progress? First, let me say that I am not in any way involved in the defining of TPC benchmarks, and have only been peripherally involved in TPC bench- marking at HP. If you really want to know what TPC thinks the benchmarks are good for, you should ask them. As I understand it, the TPC-A numbers measure automatic teller performance. It assumes human users at terminals, some of whom are active and some of whom are not. The active users make requests at a rate reflective of the speed at which humans work. This means that in order to get a large TPC-A number, the system must support a large number of active and non-active connections. These connections consume resources on the host machine and reduce its throughput. For an environment where the end-users are not humans but machines, these constraints may not be appropriate. A network of remote sensors, for example, might generate the same number of transactions with far fewer connections. The same might be true for automated controllers on a factory floor. In such cases, the TPC-B number may give a better picture of the capacity of the machine. --bob-- renglish@hplabs Not a spokeman for anyone, including myself.