Path: utzoo!attcan!uunet!dino!ux1.cso.uiuc.edu!ux1.cso.uiuc.edu!m.cs.uiuc.edu!p.cs.uiuc.edu!gillies
From: gillies@p.cs.uiuc.edu
Newsgroups: comp.arch
Subject: Re: SPECmarks
Message-ID: <76700146@p.cs.uiuc.edu>
Date: 21 Feb 90 17:34:40 GMT
References: <7393@pdn.paradyne.com>
Lines: 29
Nf-ID: #R:pdn.paradyne.com:7393:p.cs.uiuc.edu:76700146:000:1357
Nf-From: p.cs.uiuc.edu!gillies    Feb 20 22:02:00 1990


> It will be difficult for such a benchmark to avoid caching effects, with
> all the sizable/big/humongous caches that are now appearing.

Hold on a minute.  I think some people are making blanket statements
about caching and performance that aren't true.  

A good benchmark should give you *some* idea of what a cache fault
costs.  But a trivial 5-line program that will cause continuous cache
faults -- just allocate a monster piece of memory and access random
words for a while.

Now maybe a good benchmark should test for a separate instruction
cache, if the machine has it.  But having a huge benchmark is THE
WRONG answer, since it is not a time-independent solution.  You need
to find a way to test the instruction cache without depending upon the
fact that today's caches are 32-128K, since tomorrow's caches may well
be 4-8 megabytes, and next years may be 32-64 MEGABYTES.  Clearly,
SPEC would be outdated very quickly.  It may even be true that
self-modifying code is the best way to do this.

I don't disagree that you need to test a spectrum of operations; the
SPEC benchmark is good in this respect.  But keeping SPEC large is a
poor way to test for caching affects.


Don Gillies, Dept. of Computer Science, University of Illinois
1304 W. Springfield, Urbana, Ill 61801      
ARPA: gillies@cs.uiuc.edu   UUCP: {uunet,harvard}!uiucdcs!gillies