Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!usc!apple!veritas!amdcad!sun!aeras!jt From: jt@aeras.uucp (J T McDuffie) Newsgroups: comp.benchmarks Subject: Re: Price/Performance figures for Number-Crunching Summary: SPEC Quote Message-ID: <1991Mar23.192405.7668@aeras.uucp> Date: 23 Mar 91 19:24:05 GMT References: <1991Mar20.104926@IASTATE.EDU> <1991Mar21.000302.10103@convex.com> Reply-To: jt@aeras.UUCP (J T McDuffie) Organization: Arix Corporation, San Jose, CA. Lines: 38 In article <1991Mar21.000302.10103@convex.com> patrick@convex.COM (Patrick F. McGehearty) writes: > >So, I would argue that you should not distain the old benchmarks for >"benchmark rot", but be wary of any single number to characterize an >architecture. Look at several benchmark results (as many as you can find), >and cross check the different machines on the different tests. When a large >variety of tests show machine A to be twice as fast as machine B, then you >can have more confidence of your result than when a single benchmark shows >such a result. If the results vary, then understanding why will lead to >better understanding of how the machines would work in your environment. > If you receive the SPEC newsletter you'll find that they have started to utilize multiple figures of merit to describe any given machine, feeling that THERE IS NO SINGLE FIGURE OF MERIT. Thus SPEC has the SPECmark, SPECthruput, SPECint (SPECmark considering INTEGER only CPU benchmark components), and SPECfp (SPECmark considering FLOATING POINT only CPU benchmark components). So it would seem that at least SPEC would agree with McGehearty that you need to evaluate a NUMBER of benchmark results when attempting to characterize any given system/architecture. -- =====================================DVC===================================== My boss will disavow any statement I make - thinks I talk too much! ==> ==> // James T. McDuffie, III ==> ==> // aeras!jt@Sun.COM // Manager, Benchmark & Performance // uunet!sun!aeras!jt // Arix Corporation // (408) 432-0263 (FAX) // 821 Fox Lane // (408) 922-1879 (Voice) // San Jose, CA 95131 // My opinions are my own ========================= ============================ -- -- <=======================================================================> < Disclamer: My opinions are my own. No one else seems to want them! > <=======================================================================>