Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sun-barr!olivea!uunet!zaphod.mps.ohio-state.edu!sdd.hp.com!hplabs!hpda!hpcuhc!dhepner
From: dhepner@hpcuhc.cup.hp.com (Dan Hepner)
Newsgroups: comp.databases
Subject: Re: sql benchmarking environment (maybe tpc-b?)?
Message-ID: <2060015@hpcuhc.cup.hp.com>
Date: 23 Apr 91 23:44:10 GMT
References: <3632@dagobah.UUCP>
Organization: Hewlett Packard, Cupertino
Lines: 33

From: tim@ohday.sybase.com (Tim Wood)

>A couple of inaccuracies: the TPC-B is a documented benchmark.  Anyone
>is free to create the schema, emulate the workload on the desired 
>configuration and measure the results.  
> [...]
>The technology we use to emulate the workload is probably proprietary; 
>after all, it saves labor to know how to model accurately x00 terminals
>firing transactions at a database.

TPC-B does not require emulating terminals.

  How would it be in a vendor's interest 
>to tell everyone (ie competitors) how that's done? 

TPC-A, on the other hand, does.  It also requires disclosure as to
how this was done, so as to allow both competitors and customers
to judge whether or not the emulation was realistic. 

 Moreover, if a vendor
>publishes an implementation, it incurs a support obligation.  That can
>put it in the position of supporting the benchmarking efforts of those
>with contrary interests, who may not exercise complete impartiality
>before making performance claims about the product.

TPC specifically bars publishing results without the cooperation of
all relevant vendors involved.  It was the direct intent of TPC however 
to require that such results be reproducible (again, by either customers
or competitors) based on reading the full disclosure, although in reality 
this goal may have proven elusive, in part due to the acceptability
of software not (yet) generally available.

Dan Hepner