Path: utzoo!utgpu!water!watmath!clyde!att!occrsh!uokmax!rmtodd
From: rmtodd@uokmax.UUCP (Richard Michael Todd)
Newsgroups: news.misc
Subject: Re: Random order news
Message-ID: <1426@uokmax.UUCP>
Date: 21 Jun 88 05:14:08 GMT
References: <3543@enea.se>
Reply-To: rmtodd@uokmax.UUCP (Richard Michael Todd)
Organization: University of Oklahoma, Norman, OK
Lines: 86

Well, I don't claim to know all the details of why Usenet propagation behaves
the way it does, but I'll try.  No doubt if I really screw it up someone
will point it out....
In article <3543@enea.se> sommar@enea.se (Erland Sommarskog) writes:
>I am not any Unix hacker, nor am I any news wizard, just a ordinary
>Usenetter. I realize that what I'm asking for is not easy to achieve,
>yet I think the problem should be addressed. It is a nuisance, and
>I am not the only one to be annoyed.
>The problem: News articles seem to come in an almost random order.
...
>I got the original article. Some of the prematures were posted in 
>June, while the original stemmed from May 24th. A friend of mine at 
>Yale, told me he received them in the right order.

Happens here too, though usually not as drastically--propagation times
are shorter here.  The fact that one site saw them in the right order
is no guarantee whatsoever that others will.  

>Where is the problem?  

  First of all, a brief lecture on how netnews works.  When a site 
receives a news batch (or a single article) it takes each article, copies
it into the appropriate /usr/spool/news directory where you end up reading
it, and stores the articles in batches it prepares to feed for neighbor
sites.  (Actually, at least in C News, the article id are merely listed
to a file and the batches are made up later.  I think B News works similarly,
but I've only got ready access to a copy of C News.)  So if two articles come
in in the same batch and can go out in the same batch (batches are usually
made up to be about the same size, and there is no requirement that 
the outgoing batch size be the same as the incoming), their order will
be preserved.  
  However, if the articles arrive in different batches, there is no guarantee
that they will arrive in order.  The batches are stored up in the outgoing
uucp directory waiting transfer to the other system, but there is no
guarantee whatsoever that the current uucp software will transmit the
batches in the same order they were queued.  Although I don't know for
sure just how uucico scans for files awaiting transfer, I strongly suspect
it just opens the directory and reads each directory entry looking for 
files destined for the system (that's the simplest way to do it). Especially
on very active sites where files are being added to and deleted from the uucp
spool directory all the time, the order the files appear in in the directory
will bear almost no resemblance to the order in which they were put into the
directory.
  I believe that overseas sites like mcvax only contact uunet on weekends
so that their phone bills are lower; at any rate the two sites apparently
make contact relatively infrequently (compared to other Usenet links).  
If mcvax only gets news batches every week from uunet, by the time it
dials in there are a whole week's worth of news batches waiting for it.
And as I mentioned above, they're not going to be in any particular order.
Seeing a June 14 posting in the middle of a whole bunch from June 8 is
hardly surprising.  

>the American part of Usenet? But why did Yale get it in right order? 
Presumably they get news batches in often enough so that only, say, 1
day's worth of news is sitting in the queue getting randomly shuffled.
It all depends on how often one's site contacts its neighbors for news
(and how often the upstream sites contact *their* neighbors).

>anyone could say "we're planning improvement in the news software",
>I'd be even more grateful. (E.g. Mcvax could when taking news
>from uunet in date order, instead the order of arrival to uunet.
>The at least every batch could be correct.)

It would probably help some.  The question is, does Rick Adams really
want to hack on his uucp software?  Given that his site supports a whole
horde of uucp sites, I'm not sure I'd want to risk messing with the 
software, as long as it still worked.  For that matter, the extra overhead
involved in sorting queued uucp jobs by time may itself be too long to
do it in uucico; there's little merit in making sure the jobs are arranged
in order if the connection has timed out by the time you've finished
sorting.  
  I should mention in closing that this problem of Usenet not preserving
the order of articles is hardly new.  It's been around as long as I can
remember reading netnews (ca. 4 yrs now).  I even remember all the chaos
it caused with notes, which implicitly assumes that base notes always
arrive before followups.  (Remember "Orphaned Response"?)  Thankfully this
site no longer has to put up with notes...
  Final note: this problem is hardly unique to Usenet, as one can readily
discover by checking out one's local Fidonet Echos (an Echo is the
Fido analogue of a newsgroup).  Scrambled message ordering occurs almost
all the time on Fidonet, probably for the same reasons.
-- 
Richard Todd		Dubious Domain: rmtodd@uokmax.ecn.uoknor.edu
USSnail:820 Annie Court,Norman OK 73069 	Fido:1:147/1
UUCP: {many AT&T sites}!occrsh!uokmax!rmtodd
"MSDOS is a Neanderthal operating system" - Henry Spencer