Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!husc6!endor!reiter
From: reiter@endor.harvard.edu (Ehud Reiter)
Newsgroups: comp.arch
Subject: Re: Towards A Meaningful Performance Measure
Message-ID: <3122@husc6.UUCP>
Date: Fri, 6-Nov-87 08:59:38 EST
Article-I.D.: husc6.3122
Posted: Fri Nov  6 08:59:38 1987
Date-Received: Sun, 8-Nov-87 13:39:33 EST
References: <861@winchester.UUCP> <2993@phri.UUCP>
Sender: news@husc6.UUCP
Reply-To: reiter@harvard.UUCP (Ehud Reiter)
Organization: Aiken Computation Lab Harvard, Cambridge, MA
Lines: 29
Keywords: benchmarks

In article <884@winchester.UUCP> mash@winchester.UUCP (John Mashey) writes:
>Unfortunately, one can do the whole song-and-dance about
>	a) No one number is enough
>	b) Benchmark the real applications
>	c) Individual machines vary a lot
>and somebody who hearsthat, and intellectually accepts that, will STILL
>then ask you: "But how many mips is it?"  In particular, the trade press
>ends up doing this by default, as they end up showing mips-ratings,
>even in the same issues that contain good explanations of why this is only
>a gross approximation.

There's no question that the trade press is in love with MIPS, and that people
who should know better still ask about MIPS.  The point, though, is that since
it is impossible to define a single figure that measures performance, we, the
(enlightened?) readers of comp.arch, should not waste our time trying to do so.

We should also realize that MIPS do have one great advantage over newer and
more "scientific-sounding" performance measures, and that is that since most
people do realize that there is something funny about MIPS, they take MIPS
figures with a large grain of salt.  I have a feeling that this is less true
of the newer and more "scientifically defined" benchmarks like Dhrystone,
which many people take much more seriously than MIPS, even though Dhrystone
suffers from the same fundamental problem as MIPS that single figures are
meaningless (not to mention Dhrystone's numerous technical difficulties, which
have been discussed at length on comp.arch).

					Ehud Reiter
					reiter@harvard	(ARPA,BITNET,UUCP)
					reiter@harvard.harvard.EDU  (new ARPA)