Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!uunet!microsoft!gordonl From: gordonl@microsoft.UUCP (Gordon LETWIN) Newsgroups: comp.os.os2 Subject: Re: HPFS performance Summary: HPFS Performance Message-ID: <52089@microsoft.UUCP> Date: 7 Mar 90 21:19:13 GMT References: <22705@unix.cis.pitt.edu> Distribution: usa Organization: Microsoft Corp., Redmond WA Lines: 66 In article <22705@unix.cis.pitt.edu>, pdccs@unix.cis.pitt.edu (Patrick D. Champion) writes: > > I have just finished reading the OS/2 column in the March edition of Byte. > I had thought that HPFS was supposed to have a 30 to 400 percent improvement > in throughput over DOS FAT. Yet, the writer often had HPFS comming out slower > than DOS FAT. In two cases HPFS came out faster. One case was negligibly > faster, the other (random read/write) was 35 percent faster. > What's going on here? Why is HPFS performing so slowly? There are several factors that effect HPFS performance. First and foremost is the buffer size. HPFS is designed to use a larger buffer pool then the FAT file system since buffer RAM is much more available when we're not constrained to a 640K address space. HPFS buffers are 2K each; this improves performance when buffering data and directories, but it also means a poorer buffer hit ratio compared to FAT which has 512 byte buffers. For a "good sized" HPFS buffer pool - say 256K, the difference is minor, but for the default buffer pool of (32K? 64K?) HPFS is impacted some. In other words, the default buffer size for HPFS is smaller than optimal. Secondly, HPFS is an installable file system whereas the FAT file system is built into OS/2. HPFS is much faster than FAT, but it takes longer for a DOSREAD call to get to the HPFS code then it does to the FAT code. As a result, the more calls you make for smaller and smaller requests, the more that HPFS's speed is masked by the OS/2 overhead. Issuing "FindNext" calls for a single name rather than several names, issuing reads for 10 bytes, etc., - these things won't be any faster with HPFS. Naturally, future releases of OS/2 will have their IFS performance improved to aleviate this. Finally, and most importantly, HPFS gains a lot of it's performance for "small, short" operations via it's lazy write mechanism. The author of the byte article may have failed to install the lazy write handler, thus hamstringing HPFS. In that same BYTE issue, towards the front of the magazine, there's a quickie review of OS/2 1.2; The author explicitly mentions setting his buffer size higher and reports major speed improvements for HPFS. I ran the benchmarks myself when HPFS was being developed and - at least, if properly configured - HPFS has significantly improved performance in most areas. But as I've said, OS/2 overhead masks improvements for small and quick operations. Another factor to consider are the reduced limitations of HPFS. HPFS is "this much" faster than FAT to open a file in a directory with 10 files, but HPFS is many many times faster opening a file in a directory with 1000 or 10000 files. Ditto random access to large files - HPFS can be orders of magnititude faster. Ditto heavy I/O systems that want to dedicate 2 megs of buffers to the disk. Sure, you say that nobody does these things. Thats right, they don't, because until now they couldn't. HPFS gives you the capability to do these things which - for some programs and some users - can be very handy. Humans will rarely want to manage 1000 files in a directory, but my mail manager program, for example, finds it convenient to have 10000 files named "1" to "10000". When he's told to display a message he finds its name in a data base file and tells the system to open file "5267". An important goal for HPFS was to remove the performance-mediated restrictions on file, directory, and disk sizes. Gordon Letwin Microsoft related to it's monitary cost, not the amount of precious "640K" space that it uses up. each::