Path: utzoo!ncc!lyndon
From: lyndon@ncc.UUCP (Lyndon Nerenberg)
Newsgroups: news.software.b
Subject: Re: news software speedup
Message-ID: <10150@ncc.UUCP>
Date: 23 Mar 88 19:06:19 GMT
References: <649@bms-at.UUCP>
Organization: Nexus Computing Inc.
Lines: 39
In-reply-to: stuart@bms-at.UUCP's message of 22 Mar 88 20:17:41 GMT

In article <649@bms-at.UUCP> stuart@bms-at.UUCP (Stuart D. Gathman) writes:

   A major problem with the current system is scanning article headers
   in many seperate files.  (And unix doesn't like big directories to boot.)
   My idea is to have 2 files per newsgroups directory (other than sub-
   directories).  All headers for a news group would be in one file
   with offsets into another file containing all articles.

   Processing incoming news would then be faster.  Programs like 'vn' would
   be orders of magnitude faster.  The only problem is 'expire'.  I maintain
   that 'expire' would still be reasonable.  It would work by reading the
   header file and writing a new version for each newsgroup.  It can seek
   past articles to be deleted while copying the ones to be retained
   to a new file.  When finished, move the new versions into place.  This
   needs to be done only one newsgroup at a time, so there is no disk
   space problem.

There is some potential for trouble on System V implementations using
this method. Most USG systems are configured with a ulimit of 1 meg.
If you exceed this limit within a particular newsgroup, articles will
be lost when the append to the file fails.

There are two solutions: 

	1) Increase the ulimit by reconfiguring the kernel, or
	2) Make rnews setuid to root, allowing it to increase
	   its ulimit value.

In many cases, 1) is impossible (or very difficult), depending on
how the vendor has packaged their USG port. Also, an administrator
may have a good reason for maintaining a small(er) ulimit value.

As for making rnews setuid to root, I have to be a bit nervous about
the idea given the recent discussion on setuid security. I would want
to look at the rnews source quite closely before implementing it in 
this manner.

Not that I'm trying to knock the idea - it looks promising.
-- 
--lyndon  {alberta,uunet}!ncc!lyndon  lyndon%ncc@uunet.uu.net