Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!watmath!clyde!burl!ulysses!bellcore!decvax!decwrl!glacier!reid
From: reid@glacier.ARPA (Brian Reid)
Newsgroups: net.news.adm,net.news.b
Subject: curious 2.10.3 efficiency situation
Message-ID: <5044@glacier.ARPA>
Date: Fri, 7-Mar-86 03:03:04 EST
Article-I.D.: glacier.5044
Posted: Fri Mar  7 03:03:04 1986
Date-Received: Sat, 8-Mar-86 22:40:11 EST
Organization: Stanford University, Computer Systems Lab
Lines: 40
Xref: watmath net.news.adm:545 net.news.b:1307

I upgraded to 2.10.3 about a month ago, and except for the "expire" bug that
has already been reported here, I am quite happy with it.

However, about a week ago I made a stupid mistake that caused all incoming
news from all of our feeds to vanish into a pit. It was 3 days before I was
able to figure out what I had done (forgotten to turn on the setuid bit on a
piece of home-grown distribution software), so we lost 3 days' worth of news.

Once I fixed the bug, I got one of our feeds (decwrl) to retransmit the 3
days' worth of news. This was 155 batches of compressed news, and thus 155
separate calls to rnews via uuxqt. This was 3 days ago.

What happened then was both horrible and fascinating. Our machine has had
load factors of 10 to 12 around the clock for the last 3 days, and as I
watch the queues and analyze things, it looks as though it is not likely to
quiesce soon. Even as I type this message, it is about midnight, I am the
only user on the machine, and the load factor is 9.95.

The increased rate of news-flow, a pulse of 3 days' worth of news shipped in
one 8-hour burst, gave our system an impulse that is not damping out.
Glacier is now loaded down badly enough that it is unable to process news as
fast as the ordinary news is coming in, and the situation is actually
getting worse rather than better. In 2 days the number of unprocessed uucp
"X." files has grown from 155 to 220.

This is an elementary queueing problem, of course. The rate at which 2.10.3
rnews (in conjunction with compress 4.0, uuxqt, /bin/sh, and all of the
other accessories) seems to be able to process incoming messages on our
VAX-750 is just barely fast enough to keep up with the rate at which
ordinary messages are arriving.  Giving it a sudden burst of 1000 articles
has saturated things, and increased some cost (perhaps the history file
search?) enough that the uucico|uuxqt|sh|compress|rnews pipeline is not able
to process messages as fast as they are arriving.

What I am going to do to fix this problem is to run news at nice -16 all
weekend; that should give it enough of an increased speed that I can drain
the queues.
-- 
	Brian Reid	decwrl!glacier!reid
	Stanford	reid@SU-Glacier.ARPA