Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!seismo!rutgers!sri-spam!ames!pioneer!eugene
From: eugene@pioneer.arpa (Eugene Miya N.)
Newsgroups: comp.arch,comp.sys.nsc.32k
Subject: Re: "Unoptimizing" Dhrystone
Message-ID: <1329@ames.UUCP>
Date: Tue, 21-Apr-87 12:59:35 EST
Article-I.D.: ames.1329
Posted: Tue Apr 21 12:59:35 1987
Date-Received: Wed, 22-Apr-87 04:20:52 EST
References: <4190@nsc.nsc.com> <951@moscom.UUCP> <2577@intelca.UUCP> <999@mips.UUCP> <312@gumby.UUCP> <1706@husc6.UUCP>
Sender: usenet@ames.UUCP
Reply-To: eugene@pioneer.UUCP (Eugene Miya N.)
Organization: NASA Ames Research Center, Moffett Field, Calif.
Lines: 53
Xref: mnetor comp.arch:1031 comp.sys.nsc.32k:100

In article <1706@husc6.UUCP> reiter@harvard.UUCP (Ehud Reiter) writes:
>In article <312@gumby.UUCP> earl@mips.UUCP (Earl Killian) writes:
>>At MIPS, we use the following programs for benchmarking and
>>architectural study:
>>	<list of 10 or so programs>
>>All of these have very different statistics.
>
> discussion about statistics.
>
>So, if the community does shift to using multiple programs for benchmarking
>(which I strongly approve of), we should consider reporting "relative
>performance" not as a single average number, but rather as a range.
>
>					Ehud Reiter
>					reiter@harvard	(ARPA,BITNET,UUCP)

Rather than prematurely throw statistical tools at the problem, why not
pick up a book on experiment design and measurement?  

Rather than use ranges or single numbers consider other ways to convey
information.  First, you have to verify things: you can use the:
has or not have characterization.  Graphics is real popular now.
Consider the work done on Faces to display multidimensional data at Los
Alamos Natl. Lab (where I'm going next week).

Consider other analyses of dynamics: Muybridge's studies on motion.
Consider that hearts are characterized by a rate, but an EKG is better
characterization of heart rhythm.  Cardiologists can get away from
describing hearts solely in terms rate.  [Don't argue that heart rate is
a good over all characterizer, it isn't.]  We have to get away from our
infatuation with scalars, it will also stump the marketing types.

I like Earl's elaboration on whetstones.  It reminds me of a time I was
skiing two weeks ago: there was no snow at base of North Star, but up
higher on the mountain was a snow gauge with 5.5 feet of snow
registering in a flat, unskied, tree sheltered area.  So what's the
coverage at this ski resort?

No single figures of merit.  (A tee shirt I am making for our team).
I will try to have extras at SIGMETRICS 87 and PERFORMANCE 87.
Paper coming soon [I'm still late].  Stop some of this B.S.

From the Rock of Ages Home for Retired Hackers:

--eugene miya

  Not Ready for Prime Time Performer

  NASA Ames Research Center
  eugene@ames-aurora.ARPA
  "You trust the `reply' command with all those different mailers out there?"
  "Send mail, avoid follow-ups.  If enough, I'll summarize."
  {hplabs,hao,ihnp4,decwrl,allegra,tektronix,menlo70}!ames!aurora!eugene