Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Path: utzoo!mnetor!seismo!mcnc!ece-csc!ncrcae!ncr-sd!sdcsvax!darrell From: darrell@sdcsvax.UUCP Newsgroups: mod.os Subject: Re: Performance analysis of computer systems Message-ID: <2686@sdcsvax.UCSD.EDU> Date: Wed, 4-Feb-87 03:07:12 EST Article-I.D.: sdcsvax.2686 Posted: Wed Feb 4 03:07:12 1987 Date-Received: Sat, 7-Feb-87 09:35:28 EST Sender: darrell@sdcsvax.UCSD.EDU Organization: MIPS Computer Systems, Sunnyvale, CA Lines: 52 Approved: mod-os@sdcsvax.uucp In article <2683@sdcsvax.UCSD.EDU> ucbvax!ames-pioneer.arpa!ames!eugene@sdcsvax (Eugene Miya N.) writes: >I am a direct measurement type for several reasons: >1) I believe in the Missouri principle (show me) >2) I and many other have been burned by people's claims for hardware >3) Simulation has many draw backs: call them limitations or tradeoffs, >you don't trade off in reality. >4) Direct measurement can show interactions which simulation, etc. >can't show (especially when asynchronous things take place on sequential >simulations)..... Additional comments: a) If I'm buying a computer, I'm a direct measurement type. b) If I haven't built it yet, and need to know which way to do it, I sure like simulation a lot. c) If I've built it, and I have simulations, it's awfully nice to correlate them so I can trust the simulator more, although as Gene says, some situations are very hard to simulate. To illustrate the importance and value of really good simulations, one can observe the unfortunate problem of announcing performance, and then having to backtrack horribly when the product comes out. I'm sure most people have run into some of these. Some are marketing ploys, but others appear to be cases where people simply did not even simulate typical applications. On the other hand, good simulation can be incredibly helpful, and it's hard to believe people are serious about designing new computers without doing good simulation. As an example, this last summer, we started to be able to run much larger programs than we'd been able to simulate, and also more multi-tasking stuff, on M/500s. Although the performance of the simulated benchmarks was as expected, the bigger ones were not quite the 5X 11/780 we were looking for. There were a bunch of possible ways to fix this. Our performance gurus simulated the alternatives, which led us to the simplest improvement: double the I-cache from 8K to 16K [costs about $50 and was fairly straightforward]. It would have been difficult to quickly pick the correct choice without such simulation, since a whole bunch of other proposals were also considered. Coincidentally, a prospective customer was in benchmarking the week new boards were to appear. They ran a large benchmark on the 8K machine, and address traces were also captured, which when run thru the simulator, predicted the results they got. Our gurus then fed the 16K size to the simulator, and gave the customer the simulated numbers. The next day, the new boards arrived, and they re-ran their benchmarks, which matched the predictions within about 1%!! Needless to say, they were surprised. Of course, we weren't surprised at all :-) [UNIX timing granularity isn't that good: if you can get within a few tenths of a second, you're lucky!] Bottom line: simulation is necessary, but not sufficient. -- -john mashey DISCLAIMER: UUCP: {decvax,ucbvax,ihnp4}!decwrl!mips!mash, DDD: 408-720-1700, x253 USPS: MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086