Path: utzoo!news-server.csri.toronto.edu!rutgers!bellcore!uunet!news.larc.nasa.gov!grissom.larc.nasa.gov!kludge
From: kludge@grissom.larc.nasa.gov ( Scott Dorsey)
Newsgroups: comp.arch
Subject: Re: Novice question:  measuring speed
Message-ID: <1991Mar14.134319.22796@news.larc.nasa.gov>
Date: 14 Mar 91 13:43:19 GMT
References: <645@ssdc?> <3516:Mar1319:50:3291@kramden.acf.nyu.edu>
Sender: news@news.larc.nasa.gov (USENET Network News)
Reply-To: kludge@grissom.larc.nasa.gov ( Scott Dorsey)
Organization: NASA Langley Research Center
Lines: 38

In article <3516:Mar1319:50:3291@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes:
>In article <645@ssdc?> jbasara@ssdc (jim basara) writes:
>> Could someone please
>> provide me with a descriptive explaination of why MIP ratings are not a 
>> good way of comparing processing speed for RISC machines as opposed to MFLOPS?
>
>I've got this Turing machine that runs at 5000 MIPS. That's right, it
>can move the tape back and forth 5 *billion* times a second. Impressed?
>You shouldn't be: it takes so many instructions to get something done on
>a Turing machine that the MIPS measurement is pointless.

Actually, I do have a machine with a single instruction (subtract and skip if
borrow) that I built as a project for a computer architecture class.  It runs
at about 220 MIPS because it has no instruction decode time, but it isn't
very efficient at all.

This is one of the reasons that MIPS is a totally useless measure ("Meaningless
Index Promoted by Salesmen").  The other problem that we come to is that with
the new RISC machines, the actual MIPS ratings are exceedingly high, much more
so than the performance would indicate if you used CISC processors as a 
baseline.  If you printed them, people wouldn't believe them, so the marketing
people came up with MIPS-equivalent numbers.  VUPS for example ("Very Useless
Promotional Specification") is supposed to measure the compute power of a 
machine in terms of 11/780's.  It's calculated differently by every company,
and is often promoted as being a "MIPS" number.

MIPS and MFLOPS are both utterly useless because they don't have any 
information about the instruction mix.  I can execute a program full of NOPs
a lot faster than a program full of DIVs, and that's okay.  Your instruction
mix won't be the same one the marketing folks use, I can assure you.

Intel takes n benchmarks, averages the result, and points out how much faster
their machine is over some other machine.  But then, they don't have a very
realistic mixture of tasks in their benchmarks, and each benchmark is not
weighted to simulate a "typical" job mix.  This makes it completely useless.
But then, Intel also used to have an ad with a benchmark they had hoked up
to run faster on an 8080 than on an IBM 370/168.
--scott