Path: utzoo!utstat!helios.physics.utoronto.ca!news-server.csri.toronto.edu!clyde.concordia.ca!uunet!wang!fitz
From: fitz@wang.com (Tom Fitzgerald)
Newsgroups: news.software.b
Subject: Re: Faster batching in B-news
Keywords: sort
Message-ID: <1990Mar15.051023.10490@wang.com>
Date: 15 Mar 90 05:10:23 GMT
References: <37@cimcor.MN.ORG>
Organization: Wang Labs, Lowell MA, USA
Lines: 28

mike@cimcor.MN.ORG (Michael Grenier) writes:

> Could somebody gives this try?
> The following diff to batch.c forces it to build each batch
> (actually, the entire sendbatch run) with the articles
> sorted by newsgroup.

I tried this (using a script with 'sort', outside news) a long time ago and
found no measurable difference.  I was pretty depressed, it sounded like a
great idea to me too.  I rationalized that the disk was so fragmented that
there was almost no locality of reference; and that the working set of data
was so large that things in the working set were being paged out during
batching anyway.  (This was all running on a fairly feeble machine, maybe
people with more memory might have better luck).

Besides not helping any, this greatly increases the chance of reordering
articles so that a followup to an article precedes the article itself,
which is confusing to readers on the receiving end.

The only thing I found that consistently increased batching performance
was increasing the size of individual batch files.  Boosting batch size
from 50K to 100K (compressed) boosted throughput by like 30%-40%, because
compress and uux are forked half as many times, and the compression ratio
is noticeably better.

---
Tom Fitzgerald      Wang Labs           fitz@wang.com
1-508-967-5278      Lowell MA, USA      ...!uunet!wang!fitz