Path: utzoo!attcan!uunet!ncrlnk!ncrcae!hubcap!gatech!bloom-beacon!tut.cis.ohio-state.edu!osu-cis!att!pacbell!well!brecher From: brecher@well.UUCP (Steve Brecher) Newsgroups: comp.sys.mac Subject: Re: Reeks on Disktimer (LONG) Message-ID: <7595@well.UUCP> Date: 8 Nov 88 22:10:27 GMT References: <698@mouse.UUCP> Reply-To: brecher@well.UUCP (Steve Brecher) Organization: Software Supply, Sunnyvale, CA Lines: 122 In article <698@mouse.UUCP>, lad@mouse.UUCP (Larry Deleski) writes: > This is a file which I downloaded from the MouseHole BBS about a year ago. > At that time, Jim Reekes wrote an article on Disktimer, our beloved disk > performance benchmark, and essentially exposed Disktimer for was it was > (and still is), a fraud. I am the author of DiskTimer II. I am also the author of several Macintosh SCSI disk drivers, and of the firmware for several (non-SCSI) disk controllers. It is disheartening to see the republication of an article (Reekes's) which contains so much error and which I rebutted about 18(?) months ago. How a simple program like DiskTimer II, whose publication included source code, and whose user interface includes appropriate caveats about interpretation of the results could be "a fraud" is beyond me. > I would like to take this opportunity to point out an error in something I > said earlier. I stated that Disktimer did not use the Device Manager. It > does, but not in the same way a typical SCSI driver does. What Disktimer > fails to do is use the FileManager, something every driver has to do. DiskTimer II uses the disk's device driver, via _Read and _Write traps (PBRead and PBWrite) for all disk access. No device driver uses the File Manager. The File Manager uses the device driver. Applications call the File Manager which calls the Device Manager which calls the device driver which either calls the SCSI Manager or accesses the SCSI hardware directly (the latter likely to run into compatibility problems). > Please read the following [article by Reekes] and decide for yourself. > Reekes *does* know what he's talking about. Reekes has no idea of what he's talking about with respect to several issues. The following quotations indicated by ">>" are from Reekes. >> ...the Macintosh only makes single-block requests. False. The File Manager makes both single- and multiple-block requests. Reekes reiterates this error several times and bases conclusions on it. >> ...the chain of command from application to hard disk[:] >> Application>>FileManager>>DeviceManager>>DeviceDriver>> >> SCSIManager>>HDController>>HardDisk This is correct. I quote it to call it to Deleski's attention. >> DiskTimer uses the Macintosh's internal SystemTicks to calculate its >> resulting times. SystemTicks are in 1/60th of a second and DiskTimer >> is attempting to measure times in 1/1000th of a second! (ie: A typical >> 20MB drive's access time is 65ms and transfer rates are greater than >> 1MB/sec.) >> ... >> As an analogy, using ticks tomeasure a 32k read/write, would be like using a >> car's odometer to measure lengths of feet. The analogy is correct, and that's why DiskTimer II does not attempt to measure intervals such as those Reekes claims it does. DiskTimer II measures the total elapsed time to do a sequence of operations. The times measured are those reported to the user, which are usually on the orders of 10-100 deciseconds (60-600 60hz system clock interrupts). Reekes quotes Inside Macintosh to the effect that the tick count may sometimes be inaccurate by more than one tick. But the only cause of such inaccuracy is disabling of interrupts due to a continuous source of interrupts. This does not apply to DiskTimer II operations if the user follows on-screen instructions. >> DiskTimer does 200 read/write tests and averages them to the nearest >> 60th of a second. For each 100 read tests and write tests, DiskTimer II reports the *total* interval in elapsed ticks, divided by 6 (it reports deciseconds, for historical reasons). It does not attempt to measure the timing of single operations, and does no averaging. >> Most importantly, DiskTimer should never be run on a disk containing >> user data. ... If you were to reset, power off, or get the bomb dialog >> during a DiskTimer test, then chances are you've lost data. I quote this for your amusement. By this criterion, one should never run any software which writes to the disk. >> [The] seek test does not consider rotational latency. If the disk platter >> were required to complete an additional rotation just to bring that second >> block under the disk heads, then it would add additional time to the seek >> results. Rotational latency is a component of access time. As stated in the source code comments, "to a avoid constant sector skew between I/Os, which might be unfair to some drives, there is a varying delay between I/O requests. The delay is obtained via a pseudo-random number generator, using the same seed on each run." >> DiskTimer disclaims any comparisons with real world performance. DiskTimer II tells the user that "The results to not translate directly to perceived througput [sic] of typical Mac activity." Moving from response to my thoughts on this matter: DiskTimer II is a simple program that can be useful in judging the relative performance of disk subsystems on a limited set of tasks: transfers of 24KB of data to and from the disk, and a series of single-block transfers located 2048 blocks apart. The latter test is invalidated, as noted by Ephraim Vishniac, by cacheing controllers or drivers. Implementing a "real-world" benchmark of disk performance is a time-consuming task and requires a disk dedicated to the purpose, i.e., one that contains no data that needs to survive the test. Such a test requires that the exact same files be written to each disk under test and then the exact same operations performed in the same order. Only a laboratory has the resources to set up such tests. DiskTimer II was provided as a simple, non-destructive test that any user can run. It should not be relied upon as a complete measurement of disk subsystem performance. Its measurements, where not invalidated by special considerations, provide a few data points that may sometimes be useful to some people. I would have no trouble believing that a superior tool was available (I have not looked at SCSI Evaluator). But there's a lot of ignorance about the technicalities of disk performance testing, so be careful about accepting anyone's results unless you fully understand the test techniques.