Path: utzoo!attcan!utgpu!watmath!att!tut.cis.ohio-state.edu!ucbvax!bloom-beacon!athena.mit.edu!tytso
From: tytso@athena.mit.edu (Theodore Y. Tso)
Newsgroups: news.software.nntp
Subject: Idea for nntpd
Message-ID: <14506@bloom-beacon.MIT.EDU>
Date: 22 Sep 89 01:02:12 GMT
Sender: daemon@bloom-beacon.MIT.EDU
Reply-To: tytso@athena.mit.edu (Theodore Y. Tso)
Organization: Massachusetts Institute of Technology
Lines: 30

I've got an idea for nntpd, and like to know what other people think about it.
Is it dumb? unnecessary? implemented already?

The basic problem is that if you are a well-connected site, two or three hosts
might be able to simultaneously blast with you with more or less the
same list of articles.  Since articles that are already processed are
quickly skipped over, what tends to happen is that the two or three
(or more!) sending sites quickly get syncronized and begin offering
the same articles almost simultaneously.  What then happens is the
nntpd's on your site get the offer and spawn off rnews processes,
where all but one of the offered articles get punted.  This
syncronicity causes a lot of unnecessary CPU crunching.

Now, this problem can be solved by sys file hacking, but that tends to
decrease redundancy.  So how about the following solution:  after
nntpd determins that an article isn't in the history file, it tries to
creat() some lock file based on the message-id in /tmp.  If it
succeeds, it writes its pid to that file and runs rnews.  When the
rnews finishes, the lock file gets unlinked.  If it fails, it checks
the pid in the file to see if the process still exists.  If it doesn't,
it assumes something has gone wrong and runs rnews anyway.  If the
process specified in the lock file exists, then we assume that the
article is being processed by another file, and it can be skipped
(i.e., it sends 435 back to the sending host).

What do people think?  Would this be a good idea, or not?
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Theodore Ts'o				bloom-beacon!mit-athena!tytso
3 Ames St., Cambridge, MA 02139		tytso@athena.mit.edu
   Everybody's playing the game, but nobody's rules are the same!