Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10.2 9/18/84; site topaz.ARPA
Path: utzoo!watmath!clyde!burl!ulysses!gamma!epsilon!zeta!sabre!bellcore!decvax!genrad!grkermi!panda!talcott!harvard!seismo!columbia!topaz!RWK@SCRC-STONY-BROOK.ARPA
From: RWK@SCRC-STONY-BROOK.ARPA
Newsgroups: net.works
Subject: WORKS Digest   V5 #24
Message-ID: <2253@topaz.ARPA>
Date: Mon, 10-Jun-85 19:42:22 EDT
Article-I.D.: topaz.2253
Posted: Mon Jun 10 19:42:22 1985
Date-Received: Thu, 13-Jun-85 01:18:16 EDT
Sender: daemon@topaz.ARPA
Organization: Rutgers Univ., New Brunswick, N.J.
Lines: 54

From: Robert W. Kerns <RWK@SCRC-YUKON.ARPA>

    From: doug@terak.UUCP (Doug Pardee)
    Subject: Re: Ackermania
    Date: 13 May 85 18:52:35 GMT
    But now I see news postings and get mail insisting that it was
    "unfair" to multiply this effect by comparing a VAX against a Z-80A,
    because the VAX is crippled by having an excruciatingly slow "call"
    instruction.  

Sigh.  I wish people would can all the off-the-cuff conclusions from
off-the-cuff benchmarks.

		  Now, I personally think that save-everything-in-the-
    machine-because-we-don't-know-what-the-subroutine's-gonna-do type
    "call" instructions are a direct result of HLL-mania and thus are
    fair game, but I'll give the VAX the benefit of the doubt.

Actually, the VAX CALL instructions do NOT save everything in the
machine, because they DO know what the subroutine's going to do.
(That's what the entry mask on the subroutine is all about).  The VAX
calling sequence is essentially CALLEE SAVES, not CALLER SAVES.  DEC
chose this architecture after a fair bit of benchmarking and comparison,
and a lot more thought about the matter than is reflected in your
remarks.

This entire discussion has been meaningless, because nobody has
taken the time to compare apples with apples, and even when
comparing fruit with fruit, the facts have been so piecemeal to
be impede comparisons.

Just to give you an idea of what a responsible benchmark might
include:

* Always showing the comparison on all CPU's
* Presentation and analysis of the actual instruction sequences
  involved
* Statement of exactly which compiler (and which version!) is involved.
* Before making conclusions about HLL's, actually undertaking a
  survey of various compilers (like BLISS-32, etc.).
* Limiting your conclusions to what your benchmarks actually indicate.
  (ACKERMAN indicates something about the cost of function calls,
   but does NOT indicate anything about over-all "typical" performance).
* Including a comparison of the calling sequences used (since function
  calls are what's being compared here!).
* Not publishing your results until all the facts are in.

Your original benchmark was so far from being reasonable that
a dozen messages from people trying to fix the flaws haven't
been able to repair the damage.  The danger is that people
will believe your conclusions because you've done some tests,
even when what you've done is complete nonsense.

Help Stamp Out Bogus Bencharks In Your Lifetime....Wear Mittens!