Path: utzoo!utgpu!jarvis.csri.toronto.edu!rutgers!sun-barr!sun!chiba!khb
From: khb%chiba@Sun.COM (chiba)
Newsgroups: comp.arch
Subject: Re: Criteria ... [really: are N designs better than 1?]
Message-ID: <105267@sun.Eng.Sun.COM>
Date: 17 May 89 05:37:41 GMT
References: <2368@ogccse.ogc.edu> <1464@cfa.cfa.harvard.EDU> <141@dg.dg.com> <156@dg.dg.com> <658@pitstop.West.Sun.COM> <19088@winchester.mips.COM> <169@dg.dg.com> <104996@sun.Eng.Sun.COM> <173@dg.dg.com>
Sender: news@sun.Eng.Sun.COM
Reply-To: khb@sun.UUCP (chiba)
Organization: Sun Microsystems, Mountain View
Lines: 61

In article <173@dg.dg.com> uunet!dg!mpogue (Mike Pogue) writes:
>In article <104996@sun.Eng.Sun.COM> khb@sun.UUCP (Keith Bierman - SPD Languages Marketing -- MTS) writes:
>>>
>>>  My preference is to have a third party do the numbers (no fudging allowed!).
>>
>>Typically no understanding of the machine either. In former...
>
>  Yes, I agree that it is important to get somebody knowledgable to run
>the benchmarks.  Just any third party will not do (sorry I didn't make this
>clearer).

My point is that very few third parties can do justice to a relatively
new machine ... and even for an older machine it should be someone who
lives and breathes performance studies ....

>
>>
>>It has been my experience that _most_ vendors tell the truth in these
>>tests; though they may shade it a bit :>  
>
>  Actually, some vendors shade the data quite a bit.  e.g. Intel quotes
>simulated, unrolled BLAS, no-wait-state memory, and bug-free chips in

I meant this comment about honest to be restricted to customer level
benchmarks... "standard benchmarks" have become a different ball game
entirely. But if I had a circuit simulation (or given my background a
kalman filter code) to someone (with input and sample output to verify
correctness) I find that I am generally given a truthful time back.
There are exceptions....

>their i860 benchmarks.  Also, Sun made quite a mistake in claiming that their 
>new Sparcstation 1GX could create 3D polygons twice as fast as Silicon

For the NPI there was a huge amount of handouts. For "internal" (oem,
field force, etc) use alone there was an inch binder full ... the
details of the GX board show up in multiple places... and the "SGI
typo" occurs in one place... of course the press only reads the
summaries and cannot be expected to notice that the document was
INTERNALLY inconsistent ...

>Graphics Personal Iris (Sun's new machine doesn't even handle 3D polygons).
>(NOTE:  To be fair, Sun has claimed that their error was an honest mistake, although
>it seems to me to be amazingly to their benefit to make such a mistake.)           
Thanks for giving us the benefit of the doubt.


>  Actually, as I understand it, MIPS magazine had the name before MIPS (the
>company) was created.  :)

Nope. MIPSco is several years old. This is the mag.'s first year of
operation. I expect that MIPSco will win their case in court (unless
Lemmons finally backs down).

But note that I am not a lawyer, I only play one on the net :>

Cheers

Keith H. Bierman      |*My thoughts are my own. Only my work belongs to Sun*
It's Not My Fault     |	Marketing Technical Specialist    ! kbierman@sun.com
I Voted for Bill &    |   Languages and Performance Tools. 
Opus  (* strange as it may seem, I do more engineering now     *)