Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!swrinde!mips!cprice
From: cprice@mips.com (Charlie Price)
Newsgroups: comp.arch
Subject: Re: Suggestions for SPEC 3.0 CPU Performance Evaluation Suite
Message-ID: <5181@spim.mips.COM>
Date: 27 Jun 91 19:07:06 GMT
References: <403@validgh.com> <36230004@hpindwa.cup.hp.com>
Sender: news@mips.COM
Organization: MIPS Computer Systems, Inc
Lines: 70
Nntp-Posting-Host: lloyd.mips.com

In article <36230004@hpindwa.cup.hp.com> raj@hpindwa.cup.hp.com (Rick Jones) writes:
>Everyone seems to be quite interested in measuring the performance of
>'computationally intensive' benchmarks and a few that will invlolve
>local disk I/O. Essentially what one might call stand-alone
>benchmarks. 
>
>Has anything been proposed to SPEC to measure a systems performance in
>non-standalone situations? Examples would be measures such as
>nhfsstones, and perhaps an improved ttcp (well...). The really
>interesting part would likely be DCE performance, as that seems to be
>a big step towards distributed computing. Admittedly, the problem
>becomes bigger, because the testbed is a group of systems, but isn't
>that the 'future' of computing?
>
>How much longer will all these systems remain stand-alone?
>

On 14-May-91 SPEC announced a new benchmark suite.
According to the press release,
the System Development Multi-Tasking (SDM) suite
"measures overall system performance using two workloads derived
from different application environments".

The benchmark suite consists of two different multitasking
system-level benchmarks: SDET and KENBUS1.
Both benchmarks run many tasks using various system resources
and provide a throughput measurement in scripts/hour for each
of various levels of applied load.
Both benchmarks are synthetic and run various common UNIX utilities
to provide load for the system.

SDET (Software Development Environment Throughput)
is derived from an AT&T proprietary bechmark developed
several years ago by Steve Gaede.
It applies a load to the system using UNIX commands that are
found and used in a C-based software development environment.

KENBUS1 is derived from the Monash University Suite for benchmarking
UNIX Systems (MUSBUS Version 5.2) and was originally developed by
Ken J. MkDonell.  This benchmark applies load to the system using
commands and procedures that are found in a UNIX/C R&D environment.

For more information, call or write to the SPEC administrative folks:

SPEC
c/o Franson & Hagerty
181 Metro Drive, Suite 300
San Jose, CA   95110
TEL: (408) 453-5220
FAX: (408) 453-8723


As for un-measured territory...
SPEC is very interested in new benchmarks.
From a recent SPEC Newsletter:

  New Benchmarks
  SPEC seeks possible benchmarks to be included in future releases of
  the SPEC Benchmark Suite.  Suggestions should be sent to the address
  below, along with the following information:  name of benchmark,
  author of benchmark, description (what does it measure),
  how long it runs on what class machine, and contact (name and address).

So, to everybody with great ideas about how to measure distributed
processing -- *you too* can become a part of computer history --
get those benchmarks in for consideration and maybe the next time
I answer a net query about SPEC, I will be typing YOUR name.
-- 
Charlie Price    cprice@mips.mips.com        (408) 720-1700
MIPS Computer Systems / 928 Arques Ave.  MS 1-03 / Sunnyvale, CA   94088-3650