Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!rice!uw-beaver!zephyr.ens.tek.com!uunet!convex!convex.com
From: zabaly@convex.com (Frank Zabaly)
Newsgroups: comp.sys.super
Subject: Re: Cray vs. PC Benchmark
Summary: Synthetic benchmarks and real-world performance
Keywords: Dhrystone, optimizations, benchmarks
Message-ID: <107089@convex.convex.com>
Date: 11 Oct 90 17:34:38 GMT
References: <1990Oct9.192119.4453@murdoch.acc.Virginia.EDU>
Sender: news@convex.com
Reply-To: zabaly@convex.com (Frank Zabaly)
Organization: Convex Computer Corporation, Richardson, Tx.
Lines: 44

In article <1990Oct9.192119.4453@murdoch.acc.Virginia.EDU> clc5q@shamash.cs.Virginia.EDU (Clark L. Coleman) writes:
>The September 17, 1990 issue of PC WEEK has an advertisement on pages 56-57
>from Everex. The ad claims the following Dhrystone benchmarks:
>
> Everex STEP 486/33  :  34,000 Dhrystones (19.4 MIPS)
> CRAY-X-MP/48        :  17,857 Dhrystones
> IBM 3083            :  16,666 Dhrystones
>
>Is the explanation :
	<Various possible explanations omitted>

Just thought I would quote from "Computer Architecture: A Quantitative
Approach" by Hennessy and Patterson, p.48, reproduced without permission:

	"While there are rationalizations for use early in the design,
	 there is no current valid rationale for using benchmarks and
	 kernals to evaluate working computer systems ... While kernals,
	 toy benchmarks, and synthetic benchmarks(Whetstone and 
	 Dhrystone) were an attempt to make fair comparisions among
	 different machines, use of anything less than real programs
	 after initial design studies is likely to give misleading
	 results and lead the designer astray." 

They later mention that "Dhrystone has a long history of optimizations
that skew its performance" and cite a specific string copy operation
which can be optimized by a specific compiler, resulting in a 20-30%
improvement in performance. They also mention that over 99% of real-world
string copies could not be improved by this optimization.

I don't pretend to understand the rationale behind benchmarks, but
when I read Mr. Coleman's post, I trust them even less than I did before!

>"We cannot talk of freedom unless we have private property." -- Gavriil Popov,
>Mayor of Moscow, September 11, 1990. |||  clc5q@virginia.edu

You have got that right, Gavriil old boy.
Bye
Frank E. Zabaly             UUCP:{sun,harvard,rutgers,uunet}!zabaly!convex
Convex Computer Corp                           INTERNET :zabaly@convex.com
###############################################	###########################
"Something semi-automatic                      # Caffeine, the wonder drug
 might just do the trick"                      # that works and works and
        --Timbuk3 "Standard White Jesus"       # works and works and ...
###############################################	###########################