Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!husc6!hscfvax!pavlov
From: pavlov@hscfvax.UUCP (840033@G.Pavlov)
Newsgroups: comp.misc,comp.databases
Subject: Re: RDBMS performance - disks, cache, O/S
Message-ID: <454@hscfvax.UUCP>
Date: Wed, 19-Aug-87 19:54:01 EDT
Article-I.D.: hscfvax.454
Posted: Wed Aug 19 19:54:01 1987
Date-Received: Sat, 22-Aug-87 07:26:08 EDT
References: <1170@geac.UUCP>
Organization: Health Sciences Computing Facility, Harvard University
Lines: 40
Xref: mnetor comp.misc:1073 comp.databases:427

In article <1170@geac.UUCP>, john@geac.UUCP (John Henshaw) writes:
> Every time I try to have a meaningful discussion regarding the relative
> performance of various RDBMSs, the issue of comparing disk/memory performance
> arises....
> the *incredible* multitude of machine configurations often
> defeats the task of meaningful comparison. At best, vendors are able to
> provide "best" numbers, useless to most potential customers...
> So folks, how do we compare RDBMS products meaningfully? What are the
> *important* issues, and how can one compensate for the *differences*? 
> Has anyone attempted to catalogue a "fundamental list of prerequisites"
> to benchmarking?
> 
  It depends on the reasons one wishes to benchmark.  If an intellectual exer-
  cise, then one may as well pretend to be able to interpolate for op sys, i/o,
  and cpu differences (one should also toss in the factor of new releases of 
  the dbms systems involved, since these typically have improved performance
  at the rate of 30 to 40 percent with each iteration).

  If one already has a system and an application and wishes to obtain the "best"
  dbms for it, then there is a straightforward (tho labor-intensive) way to
  accomplish it: obtain copies of the dbms's one wishes to consider,implement a
  subset of the application under each one, and benchmark the performance.  In
  doing so, vary the quantity of data involved an the number of simulated sim-
  ultaneous users involved, to get a sense of degradation under load and/or 
  size.  Implement the application using any and all optimizations, features,
  etc, available in each dbms (unless contrary to prudent operation).

  When all done, compare the results in light of overall functionality of 
  each dbms, and the methods each dbms used to achieve its performance (e.g.,
  if dbms a and b appear to be "equal" in speed, but dbms b's file structure
  requires 150-megabyte restores if something should fail, then a may be the
  better choice).

  In summary: yes, the total number of combinations from which benchmark re-
  sults may be garnered are as good as infinite.  But one can make useful
  comparisons if one has a useful purpose.  But no, there are no shortcuts
  (meaningful ones, anyway).


       greg pavlov, fstrf, amherst, ny