Path: utzoo!utgpu!jarvis.csri.toronto.edu!cs.utexas.edu!tut.cis.ohio-state.edu!zaphod.mps.ohio-state.edu!wuarchive!mit-eddie!uw-beaver!ubc-cs!alberta!calgary!ctycal!ingoldsb
From: ingoldsb@ctycal.UUCP (Terry Ingoldsby)
Newsgroups: comp.arch
Subject: Re: SPECmarks
Summary: Real world performance
Message-ID: <338@ctycal.UUCP>
Date: 22 Feb 90 19:24:15 GMT
References: <7393@pdn.paradyne.com> <76700146@p.cs.uiuc.edu>
Organization: The City of Calgary, Ab
Lines: 33

In article <76700146@p.cs.uiuc.edu>, gillies@p.cs.uiuc.edu writes:
> 
> > It will be difficult for such a benchmark to avoid caching effects, with
> > all the sizable/big/humongous caches that are now appearing.
> 
...
> A good benchmark should give you *some* idea of what a cache fault
> costs.  But a trivial 5-line program that will cause continuous cache
> faults -- just allocate a monster piece of memory and access random
> words for a while.
> 
> Now maybe a good benchmark should test for a separate instruction
> cache, if the machine has it.  But having a huge benchmark is THE

I don't think that is the idea behind SPEC.  My understanding is that
SPEC hopes to show system performance under real world conditions with
real world programs.  It doesn't really matter if the benchmark fits
in cache as long as the benchmark represents a typical size application
program.  For example, if a vendor manages to build a 10 MByte cache
(which presumably is large enough to contain a major portion of most
programs) then I see no reason to penalize the vendor by concocting
a benchmark that does random jumps to outside of the cache.

Vendors who build small caches (say 1K) will compare very poorly to
the 10 MByte system, even though processor speed might be identical.
Performance on real applications is all we should hope for from
benchmarks.


-- 
  Terry Ingoldsby                ctycal!ingoldsb@calgary.UUCP
  Land Information Systems                 or
  The City of Calgary       ...{alberta,ubc-cs,utai}!calgary!ctycal!ingoldsb