Path: utzoo!mnetor!uunet!lll-winken!lll-lcc!pyramid!prls!mips!mash From: mash@mips.UUCP (John Mashey) Newsgroups: comp.arch Subject: Re: Why is SPARC so slow? Message-ID: <1192@winchester.UUCP> Date: 19 Dec 87 21:03:10 GMT References: <6964@apple.UUCP> <8885@sgi.SGI.COM> <1115@winchester.UUCP> <6993@apple.UUCP> <1941@ncr-sd.SanDiego.NCR.COM> <36626@sun.uucp> <1162@winchester.UUCP> <167@imagine.PAWL.RPI.EDU> Reply-To: mash@winchester.UUCP (John Mashey) Organization: MIPS Computer Systems, Sunnyvale, CA Lines: 54 Keywords: SPARC, RISC, Sun-4/200, R2000, M/1000 In article <167@imagine.PAWL.RPI.EDU> beowulf!lunge!jesup@steinmetz.UUCP writes: >end include unix benchmarks for grep, diff, yacc, etc.> >John, one thing was left unmentioned: the unix benchmarks you give are >VERY dependant on filing-system implementation and, even more so, on >the disks used and how fragmented they are. I'm sure I could cause a >factor of two decrease in performance by fragmenting your disks or making you >use slower ones. Any benchmarks that rely on file-system access should either >use the same disks, preferably the exact same ones so the fragmentation is >the same (I know, impractical), or at least try to quantify these differences >and state exactly what the hardware is, avg seek times, how fragmented, etc. >Better yet is not to rely on any OS and certainly not file-system dependant >benchmarks. >If you are testing the performance of a specific system configuration (CPU, >memory, disk, OS, etc, etc) fine, do so. If you are addressing the >performance of the CPU/FPU/cache (which seems to be what is discussed in this >group), don't use those benchmarks or at least try to factor out the >peripheral/OS/whatever differences. >Just trying to adhere to your philosophy of real data wherever possible. :-) Most are perfectly valid points. Fortunately, I followed all of the rules. Unfortunately, I didn't replicate the context, so anybody who hadn't seen the earlier posting, or hadn't gotten a copy of the Brief, might be misled. The benchmarks posted were an update to the last Performance Brief, posted here, and something we distribute to anyone who asks. It spells out in EXCRUCIATING detail how things are measured, what the configurations were, why we measured what we measured, what we think each benchmark measures, etc, etc. The UNIX benchmarks listed have almost nothing to do with the file system and OS issues listed. They give user cpu times. Now, on a cached machine, user-cpu times are not necesarily independent of kernel CPU times, and kernel CPU times are certainly not independent of disks. However, the system CPU times on these range from 1-20% of the user CPU (specifically: yacc: 1%, diff: 5%, nroff:15%, grep:20%), or, about a mean of 10%, hence, any system interference is fairly small. As a guess, the influence of disk fragmentation upon the numbers presented is at worst in the 1% range, which is probably below the noise level of UNIX timing. To predict UNIX command performance (which is highly relevant to some people), it is difficult to find benchmarks that people understand, might be able to duplicate, and might relate to real performance, but which are not OS-dependent, and do no I/O. We picked ones that did a fair amount of user-level computing, compared to kernel computing, to do our best to be clear about what was, and was not being measured. (OS performance is a whole separate area, quite important also.) -- -john mashey DISCLAIMER: UUCP: {ames,decwrl,prls,pyramid}!mips!mash OR mash@mips.com DDD: 408-991-0253 or 408-720-1700, x253 USPS: MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086