Xref: utzoo gnu.gcc:50 comp.arch:6953
Path: utzoo!yunexus!geac!syntron!jtsv16!uunet!tut.cis.ohio-state.edu!cwjcc!mailrus!ames!pasteur!ucbvax!aldebaran.Berkeley.EDU!shs
From: shs@aldebaran.Berkeley.EDU (Steve Schoettler)
Newsgroups: gnu.gcc,comp.arch
Subject: Common Compilers for benchmarks (was: Re: benchmarking)
Summary: Interesting for ISA design, too
Keywords: gcc silicon compilers RTL MIPS
Message-ID: <26627@ucbvax.BERKELEY.EDU>
Date: 31 Oct 88 01:20:17 GMT
Article-I.D.: ucbvax.26627
References: <7352@wright.mips.COM>
Sender: usenet@ucbvax.BERKELEY.EDU
Reply-To: shs@ji.Berkeley.EDU (Steve Schoettler)
Organization: University of California, Berkeley
Lines: 56

In article <7352@wright.mips.COM> earl@mips.COM (Earl Killian) writes:
>I think it would be interesting to benchmark various different
>machines using gcc as the compiler.  This partially removes one
>variable: how much performance is due to the compiler and how much to
>the hardware.

That might get you somewhere, but I don't think it will what you
want to find out about the architectures, and it won't completely
remove the compiler as a variable.

Consider that gcc compiles down into an intermediate RTL description
of the original C code.  The RTL describes an idealized abstract machine,
and is designed to be abstract enough so that it can be mapped into
a variety of processors: 68K,386,370, etc.  How efficiently this RTL
description is compiled into the target machine code reflects how
close the abstract machine is to the actual target machine.

So, I think what you'll find from such a study is which machine most
closely resembles the abstract machine Richard Stallman et al 
had in mind when the RTL was designed.

I am currently working on some silicon compiler software that is
designed to give instruction set designers rapid feedback about
design choices.  One thing I have been considering is automatically
generating a gcc machine description file, which can be used to generate
a C compiler, which can be used to compile and run benchmarks. 
In other words, with the right software, you should be able to input
an instruction set and get out dhrystones (etc).

If you used only results from these C benchmarks to design your processor,
I believe that you would, after a few iterations, converge on an
instruction set that very closely matched the intermediate abstract
machine used by gcc.  

Maybe that's not such a bad idea.  Perhaps the best instruction set for
the GNU C compiler is one that uses as instructions the same basic
blocks that are described by the RTL.  Of course, you'd have to do 
something about the infinite number of registers, etc, but then
you'd have a fantastic C compiler for it!

But what about the tradeoffs and assumptions made in the design of
the abstract machine?   Certainly there were intended source languages,
and intended target machines, so how ideal is it?
Let me know if you have any answers.

I believe that using gcc for benchmarks across machines is a good idea,
as it is sort of a "constant" and removes some of the variation we do
see across machines.  I just don't know how meaningful the results
will be.

For a demonstration of how much compiler technology affects "performance",
I'd ask the MIPS folks what the difference is between typical unix code
compiled with no optimizations and with all optimizations turned on.

Steve
shs@ji.Berkeley.EDU