Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!cwjcc!gatech!galbp!wittsend.lbp.harris.com!mhw
From: mhw@wittsend.lbp.harris.com (Michael H. Warfield (Mike))
Newsgroups: news.admin
Subject: Re: Who's Messing With The Usenet ?
Keywords: article, duplication, monkey business
Message-ID: <8728@galbp.LBP.HARRIS.COM>
Date: 20 Sep 89 22:15:49 GMT
References: <2056@avsd.UUCP> <14682@bfmny0.UU.NET>
Sender: news@galbp.LBP.HARRIS.COM
Reply-To: mhw@wittsend.UUCP (Michael H. Warfield (Mike))
Organization: Lanier Network Knitting Circle - Thaumaturgy & Speculums Division
Lines: 29

In article <14682@bfmny0.UU.NET> tneff@bfmny0.UU.NET (Tom Neff) writes:
>I don't understand this complaint.  These duplicate articles are proven
>to exist by matching Message ID's, correct?  But news is supposed to
>eliminate duplicate message ID's before storage.  If this is not
>happening at some site then something is broken there.  Sites with
>multiple feeds may commonly see duplicates in the batch -- they are not
>supposed to make it to the spool directory as individual articles
>though.

	Problem is that history does not store message-id's indefinitely.
On my news engine (galbp.lbp.harris.com) I typically run a 7 day expire and
retain message-id's in the history file for 14 days (that makes for 1 - 2Meg
file and 1 - 4Meg file to accomplish that amount of latency).  Some of the
dup's I've seen are as much as four weeks out of date.  If they have and
"expires" header they go straight to junk.  I'm seeing alot of those AND just
how many of us really use that header.  I have identified some articles by
matching up message-id's on save articles with the new ones and there are some
true delayed dup's out there.  There also may be some message-id fudging as
well, I'm not real sure on that.

	I'll try posting somemore details as soon as my snark traps catch
some details.  Loop time is so long, though, and the incidents are so sporatic
that by the time I get hit with another barage, I've always figured the
mess has cleared itself and turned off my traps.

Michael H. Warfield  (The Mad Wizard)	| gatech.edu!galbp!wittsend!mhw
  (404)  270-2123 / 270-2098		| mhw@wittsend.LBP.HARRIS.COM
An optimist believes we live in the best of all possible worlds.
A pessimist is sure of it!