Path: utzoo!attcan!uunet!lll-winken!lll-lcc!pyramid!eric
From: eric@pyrps5 (Eric Bergan)
Newsgroups: comp.databases
Subject: Re: ORACLE on the cheap... questions   (LONG)
Keywords: Joe Isuzu II benchmark wars hypocracy con
Message-ID: <31974@pyramid.pyramid.com>
Date: 20 Jul 88 16:01:22 GMT
Sender: daemon@pyramid.pyramid.com
Reply-To: eric@pyrps5.UUCP (Eric Bergan)
Organization: Pyramid Technology Corp., Mountain View, CA
Lines: 72

In article <302@infmx.UUCP> aland@infmx.UUCP (Dr. Scump) writes:
> 
>NO KIDDING!
>I think you will have a tough time defending this position to anyone who
>has seen your ads in UNIX WORLD. (For example, the June '88 issue, page
>10).  This ad claims to depict a "benchmark" in which Oracle claims a
>resounding performance victory over Informix.

	Just to add a little fuel to the fire...

	The latest ComputerWorld (dated July 18th) has a two page
Informix ad. In it, it compares revenues for Informix, Oracle, and
INGRES, presumably in some segment of the UNIX marketplace. I'll leave
this graph to the financial people.

	But it also includes a bar graph of "Informix", "Competitor #1",
and "Competitor #2", which is labeled a TP1 benchmark study, and 
labels the y axis "transactions per second", without any actual numbers.
Informix is shown to be about 2.5 times competitor #1 and 2 times
competitor #2.

>Some highlights:
>   1) regarding your gripe about "you *fail* to mention what versions 
>      were being compared in the benchmark":  in this ad, you run Oracle
>      5.0 (current version) against Informix version 2.0.  Version 2.0
>      is OVER TWO YEARS OLD.  The current version for OVER A YEAR before
>      this ad ran was 2.10.00 (now 2.10.03 is current).  There were also 3
>      other releases in between, including a major enhancement release
>      (2.00.05) which came out in December *86*.

	Ditto for the new Informix ad. No mention of what versions
of any of the products.

>   3) The ad states that the benchmark is an "industry standard" benchmark,
>      but DOES NOT NAME IT.  It doesn't even name the PRODUCTS used; it
>      could have been compiled embedded SQL against an interpreted ACE
>      report, for all the ad says.

	Ditto for the new Informix ad. No mention of what products
were used.

>   4) The ad photo depicts a 3B2, although the benchmark was supposedly run
>      on a Sun, implying that the performance stats apply to their "fire
>      sale" 3B2 port.

	Ditto for the new Informix ad. No mention of what platform was
used for the test results.

	The point I am trying to make is that I don't think you should
necessarily hold an individual employee of a company responsible for
their ad campaigns unless that person actually did provide the copy
for the ad, or approved it. Certainly a one or two page ad is not
going to contain the same level of depth that a performance brief would
have. Yes, I would like to see some more reality injected into the
benchmark wars, but given their very nature, I don't think that is going
to happen in the near future. Nor is it something new - look at the
current DEC/IBM war over IBM's RAMP-C benchmark - that fight has been
going on for several years.

	Could we please tone down the attacks in this group? If you
want to post performance briefs on tests on your respective products,
I think all of us would like to read them. But holier than thou arguments
really don't get us anywhere. We all know that there are benchmarks
that Informix wins, others that Oracle wins, others that INGRES wins,
others that Sybase wins, ... Further, each of the products is constantly
evolving and improving both functionality and performance. I work with
the R&D and porting groups at most of the database companies, and believe
me, there are definite strong points to each of the DBMS's. There are
also weak points, and in every case that I know of, the vendor is aware
of these and working on improving it.

	We now return you to the your regularly scheduled show, "Battle
of the Database Benchmarks".