Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!utgpu!water!watmath!clyde!rutgers!sri-spam!ames!amdahl!chuck
From: chuck@amdahl.UUCP
Newsgroups: comp.arch
Subject: Re: Benchmarking ...[really Nelson benchmarks]
Message-ID: <7261@amdahl.amdahl.com>
Date: Tue, 26-May-87 20:51:27 EDT
Article-I.D.: amdahl.7261
Posted: Tue May 26 20:51:27 1987
Date-Received: Thu, 28-May-87 01:30:36 EDT
References: <410@winchester.UUCP> <3490003@wdl1.UUCP>
Reply-To: chuck@amdahl.UUCP (Charles Simmons)
Organization: Amdahl Corp, Sunnyvale CA
Lines: 47
Summary: System performance can be estimated from lots of simple numbers

In article <3490003@wdl1.UUCP> bobw@wdl1.UUCP (Robert Lee Wilson Jr.) writes:
>What the
>customer presumably cares about is how well the system runs his/her
>application, not the rate at which it can access memory or increment
>a register.

I'm getting a little bored with this topic so I probably shouldn't
help proliferate it, but...

There are a number of people suggesting that a holistic approach to
measuring system performance is necessary.  Being a reductionist at
heart, it seems quite likely to me that a reductionistic approach
could be viable.

The holistic approach suggests that you need to simulate the application
environment on each system that you are considering buying.  This
obviously has a number of difficulties associated with it.  Perhaps
the biggest difficulty is accurately determining what the application
environment will actually look like.

The reductionistic approach suggests that system performance can
be estimated by examining a number of pieces of the system.  In
particular, I would suggest the following pieces:

1)  raw cpu power:  how many VAX 785 MIPs does the processor get?

2)  raw I/O power:  how long does it take to get data off the disk?
Both serial access and random access should be measured.

3)  context switching speeds:  how long does it take for the system
to process an interrupt?  how long does it take for the system to
switch between user programs?

4)  compiler performance:  how well does the compiler optimize?
how slow is the compiler?  how large are the binaries produced by
the compiler?

5)  system cost.

With numbers like these I can make some ballpark estimates as
to how well a system will perform for my applications, and if nothing
else, I can quickly narrow down the number of systems that I am
considering buying.

Any other reductionists want to help me out with my argument?

-- Chuck