Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!cs.utexas.edu!uunet!mcsun!hp4nl!phigate!philtis!hd
From: hd@cft.philips.nl (Henk D. Davids @ PMSN)
Newsgroups: news.software.anu-news
Subject: Re: Speed Problem
Message-ID: <632@philtis.cft.philips.nl>
Date: 15 Sep 89 15:46:22 GMT
References: <8909140437.AA12028@uunet.uu.net>
Organization: Philips Medical Systems
Lines: 75


In article <ANU-NEWS%89091314303758@NDSUVM1> Tom Limoncelli complains about
the time that NEWS takes to start. Geoff Huston wrote an answer to that,
but I think that there is more to it.
News is slow here too (on a lightly loaded VAX-8650), and I think there may
be more than one cause for that.

1. News startup takes more elapsed time than can be explained by the
   cpu-time that is consumed, or the amount of disk I/O. My guess is that I
   sometimes have to wait for locks set by other users in the news.groups
   and news.items files.

   Possible remedy would be to have News open these files for shared
   readonly access - concurrent use will be faster because RMS does not
   have to lock records. The only cases where write access is required is
   for some news manager functions, and for posting messages by other users.
   A news session by the news manager could easily be handled as a special
   case. For ordinary users a solution would require more work: posting an
   article would then necessitate a (temporary) reopen of the news.items
   file, but the ratio between reading and posting articles is at least
   something like 1000:1, true?

2. Processing the newsrc. file takes a fair amount of time, both on startup
   and shutdown. Presently these files are written in stream_LF format,
   which is notoriously slow under VAX C.
   
   Improvement could be obtained here by using "rfm=var" in the fopen, or
   by specifying "ctx=rec". In a recent article in comp.os.vms Jerry
   Leichter wrote an informative story on this, recommended reading for
   everyone using C on VMS.
   
   Shutdown delay is also caused by the (nice) way that News creates a new
   version of newsrc.: first write the new one, then delete the previous
   and finally rename the new to newsrc.;1. Nice but time consuming;
   personally I don't care if I see more than 1 version of newsrc., nor
   whether or not the version number is 1, but that's a matter of taste.

3. The selection of a newsgroup that you did not select before in the same
   session is costly in terms of page faults. News allocates a new area for
   the complete directory of every newsgroup you select. This has the
   advantage that selecting a newsgroup for the second time is
   instantaneous, but the drawback is of course that the more frequent case
   of selecting a new newsgroup is slow. Besides, selecting many large
   newsgroups in the same run can easily make your process exceed your
   process quota (virtual page count or pagefile quota).

   Possible schemes to help here all have an effect on the functionality.
   One way is to introduce a "select/unread_only" command: select a
   newsgroup, but only include the as yet unread items in the directory.
   That way we use much less memory (and I/O on news.items), but if you
   then decide that you need to look at an item that you already read in a
   previous run you would have to reselect this newsgroup to get the full
   directory.
   Another change could be to re-use the same memory area for all selects.
   That would not save on news.items I/O, but it does help to keep the page
   faults down.

   A smaller impact on speed would have the introduction (and use :-) of a
   new qualifier "/registered_only" for the news command itself: only build
   a newsgroup directory for the registered groups. 

5. When selecting an article for reading, there is sometimes a delay of
   several seconds, even up to 30 or so, before the article is displayed.
   So far I have not been able to figure out what causes this. May be I get
   bitten by directory reads that have to wait for free space? Or can this
   too be due to locks on something?

I have to add that we feed a fairly large portion of Usenet news into ANU
News. To give an impression: our news.items file is currently 24000 blocks,
my own newsrc. file is 20 blocks. Oh and yes, we have a few restricted
newsgroups.
Please do not think that I don't like News as it is now. It is certainly in
my top-5 of VMS programs.

- Henk