Path: utzoo!utgpu!news-server.csri.toronto.edu!clyde.concordia.ca!uunet!mcsun!ukc!dcl-cs!aber-cs!odin!pcg From: pcg@odin.cs.aber.ac.uk (Piercarlo Grandi) Newsgroups: comp.arch Subject: Re: Black magic, IBM RIOS. Message-ID: Date: 14 Apr 90 13:12:26 GMT References: <1990Apr4.140713.8996@specialix.co.uk> <18208@rpp386.cactus.org> <36107@brunix.UUCP> Sender: pcg@aber-cs.UUCP Organization: Coleg Prifysgol Cymru Lines: 46 In-reply-to: phg@cs.brown.edu's message of 12 Apr 90 00:17:14 GMT In article <36107@brunix.UUCP> phg@cs.brown.edu (Peter H. Golde) writes: I tested my 386 machine on the register loop and it took 0.0 seconds (it doesn't even HAVE an inner loop). Does this mean it's faster than an S/6000? (and to think I never knew....) I hereby name this benchmark "Dhumbstone". :-) :-) :-) I agree completely :-). It is a benchmark of the dumbness of many people who don't understand the difference between benchmarks at CPU/memory level (mine, LLNL's cache busting), language level (dhrystone, whetstone, perfect), and system level (SPEC, various transactionals), and the need in any case to carefully analyze their performance profile. WHAT IS A DHUMBSTONE? Your Dhumbstone rating is a direct function of the narrowness and superficiality of your analysis of a CPU/memory (or other) level benchmark, or of your blatant vested interests in mispresenting it. At one extreme, D. Lindsay who posted the cache busting CPU/memory level benchmark, and the LLNL guys who authored it, score a flat zero rating. A honorary zero rating also goes to John Mashey, for his repeated, obnoxiously demistifying articles on benchmarking. Sorry, folks! ;-) At the other exteme, IBM marketdroids score a nearly infinite rating, because not only they published meaningless MIPS/VUPS/OOPS...! numbers based on dhrystone 1.1 compiled by a code rewriting compiler, they also did it for a product that does not need any hype to impress. I am sure they have great potential to improve their rating still more. :-( To repeat it once more: with a CPU/memory level benchmark you want that it be small, so you can analyze it carefully, and you want to look not just at run time, but also at instruction count, memory access modes, instruction ordering, etc... If you understand this, your Dhumbstone rating will unfortunately suffer and you may no longer qualify for your extra improved, newly released, RISCy CISC, state-of-the-art, multiprocessor, vectoring, caching, superscalar, highly optimized, limited time special offer, for only $9,999.95 dollars, of a complimentary subscription to this newsgroup. :-) :-) :-) (for those that have a low MhontyPhytonstone rating). -- Piercarlo "Peter" Grandi | ARPA: pcg%cs.aber.ac.uk@nsfnet-relay.ac.uk Dept of CS, UCW Aberystwyth | UUCP: ...!mcvax!ukc!aber-cs!pcg Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.uk