Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sdd.hp.com!hp-pcd!hpfcso!mike
From: mike@hpfcso.HP.COM (Mike McNelly)
Newsgroups: comp.sys.hp
Subject: Re: Floating-point benchmark comparison
Message-ID: <7370257@hpfcso.HP.COM>
Date: 14 Nov 90 17:24:13 GMT
References: <1990Nov7.194853.3570@ux1.cso.uiuc.edu>
Organization: Hewlett-Packard, Fort Collins, CO, USA
Lines: 46

> In article <7370251@hpfcso.HP.COM> mike@hpfcso.HP.COM (Mike McNelly) writes:
> >Could you please give some specifics on the machines you tested.  For
> >example, OS version, hardware configurations, and compiler version all
> >make large differences in performance.  Given the rapidly changing
> >levels of performance on all vendor's machines, it's only fair to
> >identify what you're testing and publicizing.
> >
> >Thanks,
> >Mike McNelly
> >mike@fc.hp.com
> 
> Yes, at some point, when I find some spare time, I'd like to write 
> up a more detailed article explaining the different system configurations, 
> compiler versions and the details of the computations, along with an
> interpretation of the results.  But I don't believe any of that is going to
> change the "ball-park" floating-point performance figures I have posted,
> which is what most end-users wish to know.  I can see why Mr. Mike 
> McNelly is not pleased with the posting, for no vendor likes to see its
> machine near the bottom of a performance pile, but frankly, that is not
> my problem. 

I'm not personally involved in any of the machines you've reported on
but I have seen many such lists over the years.  Few, if any, of the
authors find spare time to provide either the source code of the
benchmark in question or the machine configurations which make
substantial differences, your viewpoint to the contrary.  Small things
like floating point accelerators can change performance 2-5x, for
example.  We once found in our lab that a minor algorithmic change in
I/O policy made a substantial difference in one benchmark (but not in
overall system performance).  We're also in an era where machine
performance is increasing by roughly 2x per year on most machines so it
would be nice to know if a benchmark is comparing machines from
different time frames.

A lot of people at all of the companies represented by benchmark lists
work hard to improve the performance of their machines.  It's
disheartening to these folks when their efforts are ignored or
misrepresented.  It's also a fact that no package will make a bit of
difference if it isn't installed.

You've apparently spent a substantial amount of time on your benchmark.
Why not finish the job. Neglecting to post the details seems roughly
analogous to posting a thesis without relevant attributions.

Mike McNelly
mike@fc.hp.com