Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!utgpu!water!watmath!clyde!cbosgd!ihnp4!ptsfa!ames!amdcad!sun!decwrl!labrea!glacier!jbn
From: jbn@glacier.UUCP
Newsgroups: comp.unix.wizards
Subject: Re: Backups on Live Systems
Message-ID: <17090@glacier.STANFORD.EDU>
Date: Wed, 3-Jun-87 13:29:59 EDT
Article-I.D.: glacier.17090
Posted: Wed Jun  3 13:29:59 1987
Date-Received: Sat, 6-Jun-87 04:05:33 EDT
References: <132@dvm.UUCP> <725@aramis.rutgers.edu> <20070@sun.uucp> <867@rtech.UUCP>
Reply-To: jbn@glacier.UUCP (John B. Nagle)
Organization: Stanford University
Lines: 45


       The Multics incremental dumper, circa 1968, dealt with this problem
in a very elegant way; it may take some searching of the literature to turn
up a reference, but it's worth it.  The incremental dumper ran continuously,
backing up any file whose backup was more than some number of minutes or
hours out of date.  The design spec for the Multics file system stated that
"file system reliability should be good enough that users will be comfortable
keeping their only copy of important work in the system".

       UNIVAC's EXEC 8 Secure, with which I was at one time familiar in great
detail, did incremental dumps on live systems in a reliable way; this system,
incidentally, could write several tapes simultaneously.

       I would suggest the following rules for an on-line dumper:

    0.  The goal is to be able to recover everything not designated as 
	temporary without any loss and without any manual "cleanup" or
	"guru activity".  This is achievable and has been done for other
	systems.

    1.  Files should be normally be opened for exclusive write access while
	being dumped, so that each file dumped is in an internally consistent
	state.  Alternatively, the file should be dumped and if altered 
	during dump, dumped again later, towards the end of the dump process.

    2.  Means should be provided to request special handling for files, to
	include:

	- Do not dump (for temporary files)
	- Dump without locking (for log files)
	- Dump as a consistent group (all files in group must not change
		during dump of group, for interrelated files)
	- Signal process when dump requires file access (to indicate
	  that some task should make file or files consistent and 
	  avoid altering it during the dumping process).

    3.  The means for requesting such handling should be available to the
	owner of the file.  Most users will not use it, but some programs,
	particularly serious database systems, need it badly.
	

   These things are the difference between a system that works most of the
time and one that works reliably.

					John Nagle