Path: utzoo!utgpu!water!watmath!clyde!att!pacbell!ames!pasteur!ucbvax!decwrl!sun!pitstop!sundc!seismo!uunet!littlei!omepd!radix!jimv
From: jimv@radix (Jim Valerio)
Newsgroups: comp.arch
Subject: Checkered Benchmark History (was: NSIEVE C Source File (long))
Message-ID: <62@radix>
Date: 28 Aug 88 01:31:25 GMT
References: <1070@marlin.NOSC.MIL> <1073@marlin.NOSC.MIL> <2899@winchester.mips.COM>
Reply-To: jimv@radix.UUCP (Jim Valerio)
Distribution: comp.arch
Organization: Radix MicroSystems, Beaverton
Lines: 45

For several years now, I have been feeling guilty whenever SIEVE was quoted
as a benchmark.  You see, I'm afraid that I may be indirectly responsible
for it's use.  I'm hoping that by confessing my story, someone will be able
to tell me that it wasn't my fault, and that the benchmark has a different
etymology.  Herewith follows the story.

In 1979 (I think), I was working at Tektronix when Motorola's 68000 Pascal
compiler was first released to us.  My friend Roger Critchlow was working
at Sidereal, a company that had a 68000-based product written in assembly
language.  Roger asked me what kind of code the Motorola compiler generated,
and I responded, not so good.  He was interested in more detail, so I
offered to type in some random statements and give him a listing.

Instead of typing in entirely random statements, I typed in from memory
a preprocessing program that I had glanced at the previous evening.  That
trivial program generated a table of primes, using the compacted Eratosthenes'
Sieve.  (This table was then included as part of a sophisticated factoring
program built by Mike Penk, which was used earlier that year to pull a
factor off of Mersenne (sp?) 257 and find a 303 digit twin prime.)

I compiled that 1-page program, and printed the listing file for Roger.
The code was worse than I thought.  For example, initializing a variable
to "-1" was done by moving a "1" into a register, negating it, and then
storing that in the destination.  I dropped the listing off on Roger's
doorstep, only slighly wet from being toted through the rain.

A week later, I called Roger to verify that he'd received the listing.
He said yes, and that it was atrocious code generation.  He also suggested
that next time I don't leave the listing next to the gutter downspout.
Roger also mentioned, in passing, that his co-worker Chuck Forsberg liked
the sample program, and was using it to evaluate another compiler.
(Perhaps Chuck can comment more on this.)

The next thing I hear, a year or two later, is that Byte is using the
Sieve benchmark, contributed by Chuck Forsberg.  I think I remember seeing
the article, and noticing that the code was slightly different than my
original code.  I recall being irked that the computed function was not
described as a "compacted" sieve.

Over the years, the benchmark has changed enough that I don't see any of
my code there.  But I'm left with the guilty feeling that with an hour's
work and a soggy listing, I am responsible for one of the worst of the
often-quoted benchmarks.
--
Jim Valerio	jimv%radix@omepd.intel.com, {verdix,omepd}!radix!jimv