Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!sdd.hp.com!hplabs!hpfcso!mjs
From: mjs@hpfcso.FC.HP.COM (Marc Sabatella)
Newsgroups: comp.arch
Subject: Re: Compilers & SPECmarks...
Message-ID: <8840027@hpfcso.FC.HP.COM>
Date: 12 Apr 91 21:25:34 GMT
References: <32097@shamash.cdc.com>
Organization: Hewlett-Packard, Fort Collins, CO, USA
Lines: 19

>  2.  A number based on how many integer and floating point operations 
>      the program *actually* performs when being run.  Instead of getting
>      "credit" for the number of operations to be executed as defined by
>      the source code, "credit" is given for the runtime frequency of ops
>      in the executable.

Two obvious flaws:

a) How on earth would you measure that?  Have someone disasseble the compiled
   code and hand trace its execution, counting operations?  Or perhaps supply
   hand coded assembly versions of the program for each architecture?

b) The problem with some benchmarks is not that the compiler reduces the number
   of operations.  For instance, matrix300 requires some set number of floating
   point operations on an array.  By merely storing the array in row-major
   order rather than column major order, without changing the number of
   floating point operations (or user land integer operations), you can
   relieve the strain on the cache and TLB and get tremendous performance
   improvements.