Path: utzoo!utgpu!attcan!uunet!tank!nic.MR.NET!hal!cwjcc!ukma!tut.cis.ohio-state.edu!bloom-beacon!spdcc!ima!think!ephraim From: ephraim@think.COM (Ephraim Vishniac) Newsgroups: comp.sys.mac Subject: Re: DiskTimer II results for verious disks wanted. Message-ID: <30420@think.UUCP> Date: 4 Nov 88 20:12:50 GMT References: <1449@ucsfcca.ucsf.edu> <29@eplrx7.UUCP> Sender: news@think.UUCP Reply-To: ephraim@vidar.think.com (Ephraim Vishniac) Distribution: comp Organization: Thinking Machines Corporation, Cambridge MA, USA Lines: 110 In article <29@eplrx7.UUCP> lad@eplrx7.UUCP (lad) writes: >From article <1449@ucsfcca.ucsf.edu>, by rampil@cca.ucsf.edu (Ira Rampil): >> In article <16@eplrx7.UUCP> lad@eplrx7.UUCP (lad) writes: >>>If you're looking to compare the performance of SCSI disks >>>Disktimer is NOT the way to do it. Disktimer is not a benchmark, >>>it favors disks with a 1:1 interleave and it's results do not >>>reflect real-world disk performance. Supermac based their whole >>>marketing scheme around Disktimer, saying their disks were faster >>>because Disktimer said so. >> Could it be that DiskTimer favors 1:1 interleave disks because >> they ARE faster? Is this in dispute? >1:1 disks are faster only if the machine they're working on supports >1:1. For example, Apple claims that a 3:1 interleave is optimal for >a Mac Plus, so a disk formatted to 1:1 is going perform slower. >Disktimer's problem is that it totally bypassed the driver and made >calls directly to the low-level SCSI routines, totally ignoring the >OS. Any disk with a 1:1 interleave performed better in Disktimer no >matter what machine it was being run on. There are several other >reasons why Disktimer should not be seriously considered when >evaluating SCSI disk performance, but they are very technical in >nature. >Disktimer II: Just say NO. > Lawrence A. Deleski | E.I. Dupont Co. Larry has pointed out to me in private correspondence that my initial response was inappropriately worded. He's entirely correct in that respect, and I apologize. If ever I feel the need to insult Larry in the future, I'll take care to do it privately. That said, I'd like to correct some factual errors in Larry's posting and explain my view of disk benchmarking. DiskTimer II does not bypass the device driver. It does not use the SCSI manager traps or touch the SCSI hardware directly. Instead, it calls the device driver via the device manager. (This should be obvious from the fact that it works on non-SCSI devices. If you have any doubts, you can read the publicly available source code.) Since this what the file manager does, this is the basis for any resemblance between DiskTimer's results and actual system performance. Apple did not simply claim that a 3:1 interleave is optimal for a Mac Plus. They claimed that a 3:1 interleave is optimal for a Seagate ST225N on a Mac Plus when the disk driver operates through the Apple-supplied SCSI manager. I've tested this claim using DiskTimer and similar programs and they agree with Apple. Provided that your disk driver uses the SCSI manager, Apple's advice is applicable to a particular class of drives: MFM drives without significant cache. If you're not using the SCSI manager (and many people don't) or if your drive is RLL, has a substantial cache or FIFO, then Apple's advice simply doesn't apply. Larry's claim about DiskTimer II using "low-level SCSI routines" fails to make sense in another way. The device drivers, after all, use the low-level SCSI routines or manipulate the hardware directly. If using the low-level routines gives the fastest data transfer rates with 1:1 interleaving, why don't the device drivers get that result? So far as I know, DiskTimer II's greatest failing is that it's trivially deceived by caching, either in the disk controller or in the device driver. SuperMac's claims relied on the former case; FWB's grossly misleading advertising depended on the latter case. I'm sure other companies made equally deceptive claims, but I didn't read all the ads. DiskTimer II repeatedly reads and writes the same 18K block. Any cache over 18K immediately gains a 100% hit rate and a spuriously good score. Obviously, this isn't realistic. Does SCSI Evaluator avoid this trap? I don't know. It does give you the opportunity to compare performance through the device driver against "raw" SCSI manager calls. I expect (though I don't know) that you'll find little advantage to the raw calls. With some disks, you'll find that the device driver has the advantage because it bypasses the SCSI manager. The greatest failing with all benchmarks is that it requires a great deal of thought and knowledge to make sense of the results and apply them to "the real world." DiskTimer II boils everything down to three numbers. I personally feel that the data transfer rates are of only modest importance of a fairly wide range. That's what the programmer can tweak, though, so that's where the hot competition is. In many Mac operations, such as launching an application, much of the time is spent seeking. Seek time is entirely up to the drive manufacturer. A remarkable number of "different" Mac drives have the same mechanisms inside, so there's little competition here. (BTW, DiskTimer II's seek test is also trivially defeated by caching.) As other posters have pointed out, SCSI Evaluator gives the user a great number of options and delivers a graph with numerous data points. If you're a developer tweaking your disk driver, or a user benchmarking a variety of drives at once, this is fantastic. If you're trying to compare notes with strangers via usenet, it's very nearly hopeless. Claimers and disclaimers: I have sold disk driver software to several companies, including Jasmine (San Francisco), Colby Systems (Fresno), DataSpace (Markham, Ontario), and others that never went anywhere. I'm out of that business now. Larry Deleski is (was? what's the latest, Larry?) a principal of IDT, which owns Micah. Ephraim Vishniac ephraim@think.com Thinking Machines Corporation / 245 First Street / Cambridge, MA 02142-1214 On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?"