Xref: utzoo news.sysadmin:512 news.software.b:1076
Path: utzoo!mnetor!uunet!husc6!cmcl2!nrl-cmf!ames!oliveb!jerry
From: jerry@oliveb.olivetti.com (Jerry Aguirre)
Newsgroups: news.sysadmin,news.software.b
Subject: Re: Keeping multiple news machines in sync
Message-ID: <13176@oliveb.olivetti.com>
Date: 18 Jan 88 21:08:11 GMT
References: <6263@oberon.USC.EDU>
Reply-To: jerry@oliveb.UUCP (Jerry Aguirre)
Organization: Olivetti ATC; Cupertino, Ca
Lines: 52
Summary: Keeping article number in sync is possible

In article <6263@oberon.USC.EDU> mcooper@oberon.usc.edu (Michael A. Cooper) writes:
>	I'd like to know if anyone has any ideas on keeping several
>news hosts "sync'ed" up so as to allow users to read news on any
>machine?

To summarize; the problem is, of course,  that rnews/inews may assign
different numbers to articles when they arrive or are posted.  Even if
machine B's only feed is A it is difficult to guarantee that the
articles will be transmitted in the same sequence to B as they arrive on
A.  And then local postings on B would throw that out of sync.  Like
most things once sync is lost it will tend to get worse.  The result is
that a readers .newsrc file will not correctly list what articles have
been read if a different server is used.

Given that there is only one externally connected "master" system I can
think of one way to avoid the problem.  First is to have ALL systems
except the master (including the secondary servers) post to the master
using the nntp inews.  This avoids the problem of local postings
throwing the article numbers out of sync.

The second part of the sollution is not to use the news mechanisms to
send the articles to the secondary servers.  An obvious method would be
to use rdist to distribute the news files (including lib/active) to the
secondary servers.  The overhead of running rdist on such a large file
structure several times a day might be prohibitive.

Lacking rdist you could be very careful with your news transmission and
hope that the articles stay in sync.  Using nntp to transfer between the
systems should keep them in sync.  An occasional brute force copy from
the master to the secondaries would force them back into sync.

Another possiblity is to "find" all articles newer than the last
time, create a "tar" of those articles, copy it to the secondary, and
extract it there.  (Tar would preserve the link structure of cross
posted articles.)  The difficulty here is in handling the case where a
seconday server is down when the copy and extraction is run.

If I had to choose a method I would use find/tar during the day with an
rdist at night to handle dropped articles.  This would minimize the
overhead while staying in sync.

Having the news readers use the article numbers to keep track of what
has been read is the heart of the problem.  Unfortunately, keeping track
of article IDs consumes a lot more overhead.  Rnews has essectially the
same problem of keeping track of what it has "read" (actually received
already).  The history files using article IDs consume several
megabytes.  If everyone used consistant IDs like <nnn@site> then it
would be easy but many IDs are arbitrary strings.  For each user to
consume several megabytes just to keep track of what they have read is
probably not acceptable.

					Jerry Aguirre @ Olivetti ATC