Xref: utzoo comp.arch:4290 comp.unix.wizards:7638 Path: utzoo!utgpu!tmsoft!spectrix!clewis From: clewis@spectrix.UUCP (Chris Lewis) Newsgroups: comp.arch,comp.unix.wizards Subject: Re: How fast are your disks? Keywords: Disk I/O throughput Message-ID: <540@spectrix.UUCP> Date: 8 Apr 88 23:17:16 GMT References: <3842@watcgl.waterloo.edu> Reply-To: clewis@spectrix.UUCP (Chris Lewis) Organization: Spectrix Microsystems Inc., Toronto, Ontario, Canada Lines: 171 In article <3842@watcgl.waterloo.edu> tbray@watsol.waterloo.edu (Tim Bray) writes: >In a recent meeting we were analyzing the performance of this application that >is rather I/O bound - in particular, it performs a lot of very random accesses >here and there in large (> 100 Mb) files. Somebody said "Now, we'll assume >that Unix can do a maximum of 30 disk I/O's a second". Somebody else remarked >that that figure had been remarkably constant for quite some time. Somebody >else proposed that it was a fundamental law of Computer Science. (Of course, >we are poor peons restricted to the use of Vaxes and Suns). > >Anyhow - Presumably there are other people out there limited by this >particular bottleneck. Are there reasonably-priced unix systems out there >that do better? Are there a set of benchmarks which reliably characterize >system performance in this area? Yes. Depending on scenarios, even a Tower 32/400 can beat 30 I/O's per second. Yes to the second question, and I'll post it when it's totally cleaned up. How fast do our disks go? Well, since I'm doing some performance analysis I thought I'd show some numbers extracted from our database. Environment: Standard NCR Tower 32/400 (16Mhz 68020) without CPU caching and with some relatively slow memory. Disk: Hitachi 85Mb with 28Ms. avg seeks (moderately fast). The standard Tower figures below are using the standard NCR disk controllers (ST506). The other numbers are for a new controller we're working with (same type of disk) that uses a SCSI interface. Explanation of tests: "Random" is simply a series of lseek(... 512*random()...). read(.. bsize); Linear is simply continuous read(...bsize) and "Reread" is continous "lseek(... 0 ...); read(... bsize)" (wierd testing is so we can intuit some absolute max bandwidths). In the tables below "bsize" is the request size in bytes, "req/sec cooked" is number of requests of bsize per second thru buffer cache, "bw cooked" is bytes per second thru buffer cache. Similarly, the remaining two columns are req/sec and bandwidth for raw interface. Obviously, we should be doing this to specific files rather than directly thru the blocked or unblocked special devices. Given the amount of resources we can commit to this evaluation, and the behaviour of the caches we figure that only by running the real application on top will get the true application figures. A lot of these numbers need to be taken with a fair grain of salt - UNIX buffer cache hits (and controller cache hits) are occuring, so they don't necessarily reflect *true* physical disk speed. Just UNIX I/O throughput. For the standard Tower, the req/seq and bandwidth is true disk speed on raw. On the second environment, it's difficult to say - the controller caches blocks too. Remember, the "req/sec" figures are blocks of bsize bytes in size. So, the raw Linear test with bsize of 1/2 meg with standard tower is actually transfering about 800 blocks per second. Even buffered it's approx 50 blocks per second. Standard Tower: Random bsize req/sec bw req/sec bw cooked cooked raw raw 512 35.3103 18078 35.9298 18396 1024 16.5161 16912 35.3103 36157 2048 10.449 21399 32 65536 4096 6.09524 24966 28.4444 116508 8192 3.2 26214 25.6 209715 16384 1.72973 28339 16 262144 32768 0.864865 28339 10.6667 349525 65536 0.435374 28532 5.56522 364722 131072 0.217687 28532 2.90909 381300 262144 0.109589 28728 1.45455 381300 524288 0.0547945 28728 0.727273 381300 Reread bsize req/sec bw req/sec bw cooked cooked raw raw 512 862.316 441505 59.9049 30671 1024 546.133 559240 60.0147 61455 2048 327.68 671088 59.7956 122461 4096 170.667 699050 60.2353 246723 8192 89.0435 729444 30.1176 246723 16384 44.5217 729444 20.0784 328965 32768 10.6667 349525 10.6667 349525 65536 6.4 419430 6.4 419430 (UNIX buffer cache filled up) 131072 0.214765 28149 2.90909 381300 262144 0.108475 28435 1.45455 381300 524288 0.0547009 28679 0.727273 381300 Linear bsize req/sec bw req/sec bw cooked cooked raw raw 512 55.3097 28318 55.8036 28571 1024 27.9018 28571 52.521 53781 2048 13.9509 28571 48.0769 98461 4096 7.06787 28950 39.05 159948 8192 3.48661 28562 28.9259 236961 16384 1.74888 28653 16.9565 277815 32768 0.870536 28525 11.4706 375868 65536 0.431111 28253 5.70588 373940 131072 0.216216 28339 2.82353 370085 262144 0.104803 27473 1.41176 370085 524288 0.0547945 28728 0.705882 370085 New Controller: Random bsize req/sec bw req/sec bw cooked cooked raw raw 512 170.667 87381 157.538 80659 1024 170.667 174762 170.667 174762 2048 73.1429 149796 170.667 349525 4096 51.2 209715 128 524288 8192 25.6 209715 64 524288 16384 16 262144 64 1048576 32768 6.4 209715 32 1048576 65536 3.55556 233016 16 1048576 131072 1.82857 239674 7.11111 932067 262144 0.914286 239674 3.55556 932067 524288 0.444444 233016 1.77778 932067 Reread bsize req/sec bw req/sec bw cooked cooked raw raw 512 840.205 430185 158.3 81049 1024 780.19 798915 167.184 171196 2048 481.882 986895 146.286 299593 4096 273.067 1118481 113.778 466033 8192 146.286 1198372 81.92 671088 16384 78.7692 1290555 48.7619 798915 32768 32 1048576 32 1048576 65536 16 1048576 10.6667 699050 131072 8 1048576 8 1048576 (UNIX buffer cache filled up) 262144 0.888889 233016 3.55556 932067 524288 0.450704 236298 1.77778 932067 Linear bsize req/sec bw req/sec bw cooked cooked raw raw 512 231.481 118518 162.338 83116 1024 231.481 237037 173.611 177777 2048 115.741 237037 148.81 304761 4096 57.8519 236961 120.154 492150 8192 28.9259 236961 78.1 639795 16384 13.9286 228205 48.75 798720 32768 7.22222 236657 27.8571 912822 65536 1.83019 119943 4.04167 264874 131072 0.872727 114390 1.84615 241979 262144 0.413793 108473 0.923077 241979 524288 0.26087 136770 0.666667 349525 Sorry for the format of the tables, but this is something I hacked out of one of my statistics gathering awk scripts in a few minutes. ps: people were making comments about "2Mb/sec" controllers only transferring 1Mb per second on Multibus. Well, when the manufacturers quote bandwidths they're usually quoting instantaneous max transfer rate thru the disk interface. Eg: "Standard SCSI" is actually about 1Mbyte/sec rated that way. Then, you have to consider: - disk driver overhead - UNIX system overhead - missed rotations/interleave - actual max disk output. A standard 512 byte per sector 5.25" disk that rotates at 3600 RPM has the bytes going by the head at only 522K or so bytes/second (disregarding seeks and any controller overhead). You can't go faster than that no matter what you do. Besides, Multibus is slow.... -- Chris Lewis, Spectrix Microsystems Inc, UUCP: {uunet!mnetor, utcsri!utzoo, lsuc, yunexus}!spectrix!clewis Phone: (416)-474-1955