Path: utzoo!utgpu!watmath!clyde!mcdchg!chinet!att!rutgers!apple!vsi1!wyse!mips!mash From: mash@mips.COM (John Mashey) Newsgroups: comp.arch Subject: Re: MIPS Performance Brief 3.5, October 1988 [more than long] Keywords: benchmarks Message-ID: <7306@winchester.mips.COM> Date: 29 Oct 88 17:50:40 GMT References: <7000@winchester.mips.COM> <648@hscfvax.harvard.edu> Reply-To: mash@winchester.UUCP (John Mashey) Organization: MIPS Computer Systems, Sunnyvale, CA Lines: 25 In article <648@hscfvax.harvard.edu> pavlov@hscfvax.harvard.edu (G.Pavlov) writes: ... > But this type of result tends to color my view of single-process benchmarks, > no matter how well conceived or carefully executed. I realize that large > numbers of simultaneous "jobs" are not typical in a workstation environment, > but then again multitasking is common. > > Has there been any work addressing this ? There are a number of purachasable test suites [AIM, Neal Nelson, etc]; Workstation Laboratories includes multi-user tests in its testing; and there are PD ones like MUSBUS. It is hard enough to get industry agreement on realistic single-thread tests; it is infinitely worse on the multi-user side, as many of the benchmarks are prone to misinterpretation, and also, difficulty in making apples-apples comparisons. Some of the best ones we've seen (because people bring us all sorts), are proprietary, which is really too bad. This is an area ripe for work over the next couple years. -- -john mashey DISCLAIMER: UUCP: {ames,decwrl,prls,pyramid}!mips!mash OR mash@mips.com DDD: 408-991-0253 or 408-720-1700, x253 USPS: MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086