Path: utzoo!censor!geac!torsqnt!news-server.csri.toronto.edu!cs.utexas.edu!swrinde!elroy.jpl.nasa.gov!ames!pasteur!ucbvax!hplabs!hplabsz!renglish
From: renglish@hplabsz.HPL.HP.COM (Bob English)
Newsgroups: comp.benchmarks
Subject: Re: More issues of benchmarking
Message-ID: <6324@hplabsz.HPL.HP.COM>
Date: 6 Dec 90 19:41:23 GMT
References: <7601@eos.arc.nasa.gov> <57110001@hpcupt1.cup.hp.com> <TURNER.90Dec1
Reply-To: renglish@hplabsz.UUCP (Bob English)
Organization: Hewlett-Packard Laboratories
Lines: 30

A couple of people have raised points concerning the validity of this
one-liner, what it measures, etc.

First, many people have argued that there is too much "other stuff"
going on to evaluate what the results of this test mean and extrapolate
to other applications.  I tried using the compile-only argument to bc
and passing its output directly to dc.  The difference was measurable
only on the 825, and amounted to about 1% of the execution time.  On the
845 and 870, the difference was not measurable.

Second, many people have argued that bc(1) performance is not important,
and that the wide acceptance of this benchmark might cause vendors to
spend valuable time optimizing bc(1), when it might be better spent
elsewhere.  Despite what some people think, vendors make money by satis-
fying their customers, not by misleading them into buying deficient pro-
ducts.  Engineers and programmers will often reject optimizations that
address only a benchmark, unless that benchmark itself represents a
serious customer use.  In any case, I seriously doubt that bc will ever
displace more widely recognized benchmarks as a focus of vendor activity.

% echo 2^5000/2^5000 | /bin/time bc

HP9000/825      23.9s user      0.1 system (24.2 real, unloaded system)
HP9000/835      17.9s user      0.1 system (18.0 real, unloaded system)
HP9000/845       9.9s user      0.0 system (10.0 real, unloaded system)
HP9000/870       5.6s user      0.0 system (10.7 real, loaded system)

--bob--
renglish@hplabs
"If I were running the company, even the door handles would be cool."