Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!usc!wuarchive!uunet!sdrc!thor!scjones From: scjones@thor.UUCP (Larry Jones) Newsgroups: comp.os.msdos.programmer Subject: Re: Help with large # files in dir Keywords: Copy, XCOPY, Buffers, directory Message-ID: <132@thor.UUCP> Date: 19 Aug 90 16:42:42 GMT References: <1990Aug19.012037.18164@cs-col.Columbia.NCR.COM> Distribution: usa Organization: SDRC, Cincinnati Lines: 41 In article <1990Aug19.012037.18164@cs-col.Columbia.NCR.COM>, gchamby@cs-col.Columbia.NCR.COM (Greg Chamby) writes: > > The copy took about 2.5 days due to DOS choking on the large > number of files in the subdir. Towards the end it was copying about two > files a minute. I could watch it copy a file, scan the disk for about 30 > secs and then get the next file. Is this an inherent problem with MSDOS? > I've seen a DIR command steadily degrade after a couple thousand files as > well. Is DOS doing a linear search to get the next dir entry? I'm > wondering if there is anything one can do to speed up DOS when accessing a > huge number of files such as this. I don't believe upping the BUFFERS > would help since we're talking about a sequential copy of files. Yes, you seem to have confused the DOS FAT and directory mess with a file system. ;-) The DOS file system is TERRIBLE and handling large files, and a subdirectory with lots of files in it is itself a large file. In the case of your copy, the copy program is sequentially reading the directory to find the next file. This in itself isn't too bad, but copy then has to open the file so it can copy it. Open then has to read the directory again to find the file and it does so by reading sequentially from the beginning. That's why it get worse as it goes. Increasing the number of BUFFERS might help if you make it large enough to hold the entire FAT and directory, since then you'll be reading from memory instead of having to move the heads back and forth across the disk from FAT to directory to file. Unfortunately, DOS's buffering strategy isn't very sophisticated so it might well discard a buffer full of valuable directory or FAT information and replace it will basically worthless file data, so it might not help as much as you'd hope. OS/2's Fast File System is one of it's (few in my opinion) significant improvements over DOS. ---- Larry Jones UUCP: uunet!sdrc!thor!scjones SDRC scjones@thor.UUCP 2000 Eastman Dr. BIX: ltl Milford, OH 45150-2789 AT&T: (513) 576-2070 Rats. I can't tell my gum from my Silly Putty. -- Calvin