Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!watmath!clyde!rutgers!im4u!ut-sally!seismo!mimsy!chris
From: chris@mimsy.UUCP
Newsgroups: news.software.b
Subject: Re: Is the history file really needed anymore?
Message-ID: <5308@mimsy.UUCP>
Date: Fri, 6-Feb-87 02:51:23 EST
Article-I.D.: mimsy.5308
Posted: Fri Feb  6 02:51:23 1987
Date-Received: Sun, 8-Feb-87 05:13:44 EST
References: <5504@ukma.ms.uky.csnet> <1307@ncr-sd.UUCP> <7529@utzoo.UUCP> <414@oliveb.UUCP>
Organization: U of Maryland, Dept. of Computer Science, Coll. Pk., MD 20742
Lines: 64
Keywords: netnews history dbm

>In article <7529@utzoo.UUCP> henry@utzoo.UUCP (Henry Spencer) writes:
>>Eliminating the history file is, on the whole, a silly idea.

Eliminating it is indeed silly; replacing it may not be.  (Well,
yes, `eliminating the history file' might be a *part* of the
replacement operation....)

>>... (What makes you think that the implementation of the new concept
>>will be any better? ...)

In the words of that mortal bard, `Ay, there's the rub.'

In article <414@oliveb.UUCP> jerry@oliveb.UUCP (Jerry F Aguirre) writes:
>I see a constant series of complaints on the net about the history file
>being out of sync.  The classic case seems to be running out of disk
>space which causes the history file to wind up zero length.  It is not
>quite clear to me what changes to the news software could reliably
>handle the problem of running out of disk space.

	oldf = open("history", ...);
	newf = creat("history.new", ...);
	...
	/* write exclusively to newf, catching errors */
	/* finally: */
	if (close(newf)) ...	/* handle error */
	(void) unlink("history");
	if (link("history.new", "history")) ...
	(void) unlink("history.new");
	/* or: if (rename("history.new", "history")) ... */

The `handle errors' part is tricky; one could delay unlinking
expired articles until the entire history.new has been made
successfully, or simply wait for Intelligent help.  This also does
nothing for incoming articles.  But this is a conventional
database coherency problem, with conventional solutions.

>As far as overhead, it is hard to understand how a single call to create
>a file can have a prohibitive overhead.  That is all that is required to
>check the "history" AND make the new entry.

Name lookups are, in all Unix systems that I have seen, *the*
highest cost operation performed by the kernel.  In particular,
creating a link is hard even with cacheing, as the entire target
directory must be scanned to ensure that the file does not yet
exist.  Longer pathnames take correspondingly longer to translate,
so short message-IDs would be desirable.  In comparison, a history
file can be opened once by rnews, and one line written per article
stored with only a relatively cheap write() syscall.  (This is, of
course, only a factor if you batch articles.)

>As a directory need only be made for each N files (say 100) the
>overhead for that can not be significant.  I have heard some
>mention of "searching" directories for each received article.
>I don't understand what prompted someone to think this.

Expire.  Expire must gather information about every unexpired
article.  Without a history file, expire would have to read the
directory tree.

If you think you have a good replacement, try it.  You may find
it hard to beat C news, however.
-- 
In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7690)
UUCP:	seismo!mimsy!chris	ARPA/CSNet:	chris@mimsy.umd.edu