Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!usc!snorkelwacker.mit.edu!bloom-beacon!deccrl!news.crl.dec.com!pa.dec.com!shodha.enet.dec.com!alan
From: alan@shodha.enet.dec.com ( Alan's Home for Wayward Notes File.)
Newsgroups: comp.benchmarks
Subject: Re: bc benchmark [sigh]
Summary: No vampires.
Message-ID: <2195@shodha.enet.dec.com>
Date: 31 Dec 90 05:32:14 GMT
References: <44342@mips.mips.COM> <15379@ogicse.ogi.edu>
Organization: Digital Equipment Corp. - Colorado Springs, CO.
Lines: 83

In article <15379@ogicse.ogi.edu>, borasky@ogicse.ogi.edu (M. Edward Borasky) writes:
> My question is this: just as you and I believe that vampires don't 
> exist, do you believe that a single-number that measures a computer's 
> speed doesn't exist?

	I do not believe that a SINGLE (atomic) number exists which 
	usefully measure's a computer's speed.  The problem with a
	single number is that it may measure only one or a few aspects
	of the computer's speed.   For example: how long it takes 
	execute an instruction stream which contains 60% integer 
	divides and 39% integer multiplies and which fits into the 
	system data and instruction caches.  For some small population
	that aspect may be interesting.  For many others that is not
	a reflection of reality.

	Some people will be interested in how fast the computer does
	integer adds and subtracts.  Others, floating point arithmetic.
	Still others byte copies and compares.  And still others will
	have applications that are dominated by I/O.

	What is required are MANY (atomic) numbers, each of which
	measures one or more of the interesting aspects of a computer's
	speed.  To go along with these numbers are reasonably detailed
	descriptions of what that number measures.  That way you can
	examine your application to characterize it's use of the system
	and find the number which is the best match.  Only then is it
	safe to compare single numbers.

	Now, one thing that can be done is to take these MANY (atomic) 
	numbers and run them through some mathmatical function to get
	a single number that represents the combination of all of the
	numbers.  The function has to be carefully constructed so that
	one exceptional number (good or bad) doesn't dominate the final
	answer.

	If all bc(1)'s are created equal and you know that it's use
	of the system reflects how you use the system, then it may
	well be a good single benchmark.  I believe that using bc(1)
	is only safe as benchmark if your primary applciation is in
	fact bc(1).  If a vendor decides to make bc(1) go very fast
	at the expense of improving the performance of their compilers
	and libraries then only the people that use bc(1) will win.

    My application:

	Read 24 bytes and extract four non-well aligned fields; a 
	read/write flag, LBN number, transfer size and unit number.
	If this record is interesting, perform a variety of operations 
	some of which are:

		Look up the transfer size in a list based on the
		unit number to count how many transfers of that
		size there are (pointer chasing ending with an
		integer increment).

		Increment a counter (based on the unit number) for
		each read or write.

		Determine the absolute distance from the previous
		LBN to the current LBN (for this unit number).

		Determine what logical partition this LBN was in to
		increment another set of counters (still more pointer
		chasing for the transfer size).

	    So far the application has stressed integer add, compares
	    and moving bytes around.

		Look up the LBN in a database to determine whether or 
		not the LBN is file system metadata and what kind.

	    It just became seriously I/O bound (a large buffer cache
	    helps A LOT).

	Once all this has been done for every 24 byte record in the 
	input stream, print a summary of the results.  Most of the 
	math is done in floating point (adds with a small number of 
	multiplies and divides).

	If you like them, vampires are a nice fantasy.  As are single
	number benchmarks.
-- 
Alan Rollow				alan@nabeth.enet.dec.com