Path: utzoo!attcan!uunet!ncrlnk!ncrcae!ncr-sd!hp-sdd!hplabs!pyramid!nsc!grenley
From: grenley@nsc.nsc.com (George Grenley)
Newsgroups: comp.arch
Subject: Re: Benchmarking
Message-ID: <6899@nsc.nsc.com>
Date: 10 Oct 88 02:57:12 GMT
References: <2220003@hpausla.HP.COM> <46500026@uxe.cso.uiuc.edu> <6683@nsc.nsc.com> <6684@nsc.nsc.com> <4263@wright.mips.COM> <6729@nsc.nsc.com> <10498@reed.UUCP> <4655@winchester.mips.COM> <6868@nsc.nsc.com> <1988Oct9.011633.13259@utzoo.uucp> <4853@winchester.mips.C
Reply-To: grenley@nsc.nsc.com.UUCP (George Grenley)
Organization: National Semiconductor, Sunnyvale
Lines: 50

In article <4853@winchester.mips.COM> mash@winchester.UUCP (John Mashey) writes:
>In article <1988Oct9.011633.13259@utzoo.uucp> henry@utzoo.uucp (Henry Spencer) writes:
>
>>Note three complications:
>
>>	1. The GNU stuff is ***NOT*** public domain.

Let me apologize to the nice folks (creatures?) at GNU.  I meant that GNU source
was available & standard, but I in no way meant to imply it was in the public
domain.  Thanx to Henry at UT for pointing this out...

>>	2. It tends to be huge.
>
>Reasons why people ahve liked these as potential benchmarks are:
>	1. Although not public domain, "generally available" is much better
>	than "proprietary", which is unfortunately true for many otherwise
>	desirable benchmarks.
>	2. Tends to be huge.  GOOD!  Most of the common itneger benchmarks are
>	toys or near-toys, unlike the floating-point ones.
>-- 
>-john mashey	DISCLAIMER: <generic disclaimer, I speak for me only, etc>

I want to clarify my earlier posting about b'marks - I am primarily interested
in CPU benchmarks, not system b'marks.  This is partly because my employer
makes chips, not systems (we tried that once, and we still get mail about it),
and partly because, as a hardware engineer, I am interested in providing the
kind of info that will assist other h/w engineers in making a CPU selection
on the merits, not based on whimsy/guesswork etc.

Unfortunately, this augers against large, OS dependent benchmark programs.  I
imagine that trying to make grep run "standalone" is not trivial....

I've received a lot of email on b'marking; one individual pointed out that
the database community "scales" the size of the b'mark (i.e., size of dbase)
to the size of machine.  An interesting idea.  I think we should consider
taking some of the small integer b'marks, and "enlarge"them by having the
program call itself recursively in a non-trivial way.  Then, the test would
consist of running the program at, say, 1 through 1000 levels of recursion,
or whenver you run out of RAM.  Then, publish the performance numbers.

Comments?  I am willing to volunteer to drive this if anyone (like, f'rinstance,
someone who can code better than me) wants to help.  Maybe we'll even get it
O-f'shally blessed by IEEEEEEEE.

Regards,
George Grenley
NSC

ps to Henry Spencer - is UT still using any series 32000 iron?  If so, drop me
a note - I may have some news of interest to you.