Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sun-barr!olivea!decwrl!deccrl!bloom-beacon!eru!hagbard!sunic!mcsun!ukc!icdoc!qmw-cs!liam
From: liam@cs.qmw.ac.uk (William Roberts;)
Newsgroups: comp.unix.wizards
Subject: Re: Why is restore so slow?
Message-ID: <2880@redstar.cs.qmw.ac.uk>
Date: 28 Jan 91 15:32:29 GMT
References: <50235@olivea.atc.olivetti.com>
Sender: usenet@cs.qmw.ac.uk
Lines: 37
Nntp-Posting-Host: whitesand

In <50235@olivea.atc.olivetti.com> jerry@olivey.olivetti.com (Jerry Aguirre) 
writes:

>Has anyone done any evaluations of why there is such an extreem
>difference in speed?  Granted that creating files involves more overhead
>than dumping them restore still seems very slow.  As restore operates on
>the mounted file system it has the advantage of accessing a buffered
>file system with write behind.

>My particular theory is that the disk buffering algorithms are precisely
>wrong for restore.  By this I mean they keep in the buffers the data
>that will never be needed again and flush the data that will.  I plan to
>do some experimentation and would appreciate hearing any ideas you might
>offer.

Restore suffers from the fact that files are stored in inode-number order: 
this is not the ideal order for createing files as it thrashes the namei-cache 
because the files are recreated randomly all over the place. We reorganised 
our machine once and used dump/restore to move our /usr/spool and /usr/mail 
partitions around: /usr/spool contains lots of tiny files called things like 
/usr/spool/news/comp/unix/internals/5342 and this took an incredibly long time 
to restore. /usr/mail contains several hundred files but no subdirectories and 
restored in about the same sort of time as it took to dump. 

Restore suffers the same accidents of history as a lot of other system 
utilities. It dates back to sub 1-megabyte memory machines (maybe 64k separate 
I/D space PDPs) and so it uses pathetically small buffers. If you want to 
speed up restore, steal maybe 2 Megabytes of memory as file buffers, fill that 
with file images to be restored and then write them to disk in something 
resembling a depth-first traversal of the directory tree. This costs a little 
bit of ingenuity but doesn't involve any kernel changes.
--

William Roberts                 ARPA: liam@cs.qmw.ac.uk
Queen Mary & Westfield College  UUCP: liam@qmw-cs.UUCP
Mile End Road                   AppleLink: UK0087
LONDON, E1 4NS, UK              Tel:  071-975 5250 (Fax: 081-980 6533)