Path: utzoo!mnetor!uunet!husc6!bbn!bbn.com!rsalz
From: rsalz@bbn.com (Rich Salz)
Newsgroups: news.software.b
Subject: Re: news software speedup
Message-ID: <546@fig.bbn.com>
Date: 24 Mar 88 01:35:11 GMT
References: <649@bms-at.UUCP> <10150@ncc.UUCP>
Reply-To: rsalz@bbn.com (Rich Salz)
Organization: BBN Laboratories, Cambridge MA
Lines: 26

>In article <649@bms-at.UUCP> stuart@bms-at.UUCP (Stuart D. Gathman) writes:
>   A major problem with the current system is scanning article headers
>   in many seperate files.  (And unix doesn't like big directories to boot.)
>   My idea is to have 2 files per newsgroups directory (other than sub-
>   directories).  All headers for a news group would be in one file
>   with offsets into another file containing all articles.

This is basically how notes stores stuff.

It can get you some speed benefits, but makes updates (e.g., expire,
cancelled articles, new articles) much harder, and some things much
slower.  Currently, to add a new article, you need merely lock the active
file for a moment to bump a number.  This way you'd have to lock two big
files for a significant amount of time.

You'd also have to change the .newsrc (and all the news readers), unless
you had a third file that mapped "article number" to header file
offset.  (You need to be able to remove expired articles and close up
holes in the header file, lest it grow without bound.)

Also having one article/file makes hundreds of Unix utilities more useful
than they'd otherwise be (doing grep on a notes "text" file brings no joy;
I've done it).
	/r$
-- 
Please send comp.sources.unix-related mail to rsalz@uunet.uu.net.