Path: utzoo!utgpu!water!watmath!clyde!att!osu-cis!tut.cis.ohio-state.edu!mailrus!ncar!ames!eos!eugene
From: eugene@eos.UUCP (Eugene Miya)
Newsgroups: comp.arch
Subject: Re: Benchmarking
Message-ID: <1711@eos.UUCP>
Date: 13 Oct 88 17:06:04 GMT
References: <2220003@hpausla.HP.COM> <46500026@uxe.cso.uiuc.edu> <6683@nsc.nsc.com> <6684@nsc.nsc.com> <4263@wright.mips.COM> <6729@nsc.nsc.com> <10498@reed.UUCP> <4655@winchester.mips.COM> <6868@nsc.nsc.com> <1988Oct9.011633.13259@utzoo.uucp> <4853@winchester.mips.C
Reply-To: eugene@eos.UUCP (Eugene Miya)
Organization: NASA Ames Research Center, Calif.
Lines: 26

In article <3285@pt.cs.cmu.edu> lindsay@k.gp.cs.cmu.edu (Donald Lindsay) writes:
>First, I am solidly behind the idea that the best benchmark is the user's
>application.

This is fine if you are Livermore and devote two Crays to running many
runs of a single program.  You have problems if you run a diversity of codes.

Big programs (sorry John I can't completely) can be as deceptive as small.
Big programs have more paths to test.  It's just as blind (but in different
ways) as small programs.  Now using both you might get more.

>There is a fairly simple way to achieve these ends. Do not write a
>benchmark program: write a program which writes out the benchmark program.
> ... much deleted

I am working on this (with a company).  How much are you willing to pay?
Naw, just kidding, our ideas are too crude for a product.  We are just
writting tools to help.

Another gross generalization from

--eugene miya, NASA Ames Research Center, eugene@aurora.arc.nasa.gov
  resident cynic at the Rock of Ages Home for Retired Hackers:
  "Mailers?! HA!", "If my mail does not reach you, please accept my apology."
  {uunet,hplabs,ncar,decwrl,allegra,tektronix}!ames!aurora!eugene
  "Send mail, avoid follow-ups.  If enough, I'll summarize."