Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!apple!usc!sdd.hp.com!apollo!rehrauer From: rehrauer@apollo.HP.COM (Steve Rehrauer) Newsgroups: comp.benchmarks Subject: Re: SPECmarks Message-ID: <4de7f24e.20b6d@apollo.HP.COM> Date: 9 Nov 90 16:14:00 GMT References: <1990Nov9.012540.28546@aplcen.apl.jhu.edu> Sender: root@apollo.HP.COM Reply-To: rehrauer@apollo.HP.COM (Steve Rehrauer) Distribution: comp Organization: Hewlett-Packard Apollo Division - Chelmsford, MA Lines: 63 In article <1990Nov9.012540.28546@aplcen.apl.jhu.edu> theune@aplcen.apl.jhu.edu (Peter Theune) writes: >khb@chiba.Eng.Sun.COM (Keith Bierman fpgroup) writes: >>In article tds@cbnewsh.ATT.COM (antonio.desimone) writes: >> Could someone post a simple description of SPECmark ratings for the >> lazy outsider? One of the trade rags (Better Workstations and >> Gardens, or something) published a chart that apparently was pulled >> out of context from another source, without enough supporting material to >> make it comprehensible. > >> I gather that it's a composite measure of some kind. What are the >> components? Are the individual measures what's usually presented? >> What does it mean when I see a single SPECmark rating? E.g "the >> SPARCstation (TM) 2 family, at 21 SPECmarks..." How are the weights >> chosen? >> -- > >>the benchmarks are > >>gcc, espresso, spice, doduc, nasa7,li,eqntott,matrix300, fppp, and >>tomcatv. Footnote: these ten tests comprise the SPEC 1.0 suite (current revision is actually 1.2, I think). SPEC (the Systems Performance Evaluation Consortium) plans to release suite 2.0 sometime soon, I believe. Several new tests will be present in the new suite; there will probably be some that test I/O and overall "system-type stuff" performance, as well as new floating-point and integer tests. There's also a lesser known variant of SPEC known as the "Throughput" suite, I believe, which involves simultaneously running multiple copies of each test per processor. This is intended to provide some indication of a system's behaviour under more typical load conditions; i.e.: does the system hum for a single user process, but squeal like a stuck hog when several people simultaneously begin doing several things on it? >>The specmark is computed by computing the ratio of the machine under >>test vs a particular configuration of a VAX and then computing the > >I _THINK_ that the VAX used for comparison is a VAX-11/780 . I seem to >recall reading recently that this was the machine used for comparison. I >don't know wether or not a floating point processor was installed.... It is an 11/780. I also don't know what hardware configuration was used. If you get the full SPEC report, all that information is included, e.g.: processor type, clock-speed, OS revision, compiler revision, compiler switches used to produce the figures, amount of memory, etc. >>geometric mean of the 10 scores. Yep, that's what it is. I'm told the SPEC members originally didn't want to provide a "boiled down" number, since, like most simplistic "bottom line" figures, it really doesn't give you an accurate picture of how the system performs across the range of tests. An unbalanced system might have huge swings across the various tests. The chart, which most glossy sales adverts omit, will show this. The "SPECmark" figure doesn't. However, it was inevitable that people would want to make comfortable "X to Y" comparisons, and at least "SPECmarks" are far, far less misleading than "MIPS". -- "I feel lightheaded, Sam. I think my | (Steve) rehrauer@apollo.hp.com brain is out of air. But it's kind of | The Apollo Systems Division of a neat feeling..." -- Freelance Police | Hewlett-Packard