Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!cs.utexas.edu!sun-barr!lll-winken!ncis.tis.llnl.gov!helios.ee.lbl.gov!pasteur!ucbvax!hplabs!pyramid!eric
From: eric@pyramid.pyramid.com (Eric Bergan)
Newsgroups: comp.databases
Subject: Re: RDBMS benchmark data
Keywords: RDBMS benchmarks performance
Message-ID: <94724@pyramid.pyramid.com>
Date: 15 Dec 89 01:13:51 GMT
References: <4703@uswat.UUCP> <698@dgis.dtic.dla.mil>
Reply-To: eric@pyramid.pyramid.com (Eric Bergan)
Distribution: usa
Organization: Pyramid Technology Corp., Mountain View, CA
Lines: 53

In article <698@dgis.dtic.dla.mil> jkrueger@dgis.dtic.dla.mil (Jon) writes:
>bruce@zeb.USWest.COM (Bruce W. Hoylman) writes:
>
>>I'm evaluating RDBMS packages for potential use in a major
>>development effort in my company.
>
>And, like most people, you're focusing on performance,
>instead of functionality, reliability, vendor support,
>and programmer productivity.  Please reconsider.
>Don't be like most people.
> [...]
>You can expect adequate performance from any of these.
>Within a few years, you can expect little difference
>in performance between any of the survivors.  Please
>look at things that really matter.

	I'm afraid I disagree. I agree that if you average across all
applications, etc, that the DBMSs tend to be about the same, but the fact
is that there can be significant differences in performance on specific
facets - differences so great as to rule out the DBMS. Some examples:

	- complex query that one DBMS properly parses out, and another
	  doesn't - can mean a several order of magnitude difference
	  in performance

	- backup strategy - some products can take 8 times longer to
	  do backups than others, due to their buffering/backup strategies.
	  Can be very important if backing up larger databases

	- differences in buffer management, particularly with respect to
	  what to do about linear searches, can cause major differences
	  in performance.

	(It is true that TP1 is not a great distinguisher of performance
any more...)

	I'm sure if pointed out to the vendors, any shortcomings versus
a competitor will go on the list of things to change, but that list can
get very long, and not all items make the cut for each release. But if the
effect is a difference between a 1 second response time and a 100 second
response time, I submit that is a very important point in choosing a database.

	My suggestion is that the only way to assess the performance of
a database system for your application is to do as accurate a job as you
can at developing a benchmark which models your application and environment,
and then run it on the DBMS systems under selection. General benchmarks,
such as TP1 or DeWitt are only useful if you can understand the relationship
between the TP1 or DeWitt application model and your own.

-- 

					eric
					...!pyramid!eric