Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!cs.utexas.edu!uunet!sci34hub!gary
From: gary@sci34hub.UUCP (Gary Heston)
Newsgroups: news.admin
Subject: Re: Dynamic feeds (was Re: Why not just eliminate  ..... )
Summary: Just a couple more config files....
Message-ID: <370@sci34hub.UUCP>
Date: 21 Oct 89 01:49:50 GMT
References: <817@pmafire.UUCP> <725@lakart.UUCP>
Lines: 61

In article <725@lakart.UUCP>, dg@lakart.UUCP (David Goodenough) writes:
 
= geoff@pmafire.UUCP (Geoff Allen) sez:
= > In article <34075@looking.on.ca> brad@looking.on.ca (Brad Templeton)
= > proposes a scheme for setting up dynamic feeds
= > 
= > I think this idea's worthy of consideration.  Just how tough would
= > something like this be to do?
 
= I'd make one suggestion, partly because I believe this is the way to
= write software, and partly because it makes this job easier.
 
= Break up inews.

= [ good case made for suggestion ]
 
= I'd suggest more a dynamic "ihave - ineed" type setup. I.e.
 
= [ description of how this would work, and potential problems ] 
 
= .... - if mirror decide that we feed them
= and turn round and ask for every newsgroup on their system we're gonna flip
= out: we only have a 2400 BPS modem for communications.

Deciding who feeds who with what should be controllable with a sys file 
entry. Limit the newsgroups mirror can demand, or limit the number of 
requests a system can make. A modification of the sys file might be 
a better approach, a "have_need" file, perhaps. This could include 
any limit data needed, and could also establish a prioritized scheme.
Something like "get everything we can from the primary feed, then 
call these other feeds in this order to see what else there is".
Of course, 15,000 admins will just tell you to get a TrailBlazer,
but I'm sure you've already considered that possibility.

Potential problem with all this: batching/unbatching time lag. 
Once a "ineed" list is sent, the batch must be constructed at the
other system, transferred, unbatched, posted, and scanned by the 
dynamic feeder. I don't think this could be done in real-time, it
would probably be necessary to send a "ineed" list, and call back 
later for the batched news. Scanning 100MB takes a while....
Even if a database of articles is constructed, it'll take time to 
dig thru it and build the batches--not to mention needing space 
for it, and maintaining it. You know, this is starting to sound
a lot like a history file......  :-)
 
= I agree with Brad - this has to be possible (watch out for D news :-) ), but

Right. "Dynamic news".                                       ^

= there are going to have to be some questions answered first.

Starting with a list of volunteers to write all this stuff. Of course,
it can be broken up into pieces, and then it would be easier to
maintain, and smaller.....     :-)

= 	dg@lakart.UUCP - David Goodenough		+---+

-- 
    Gary Heston     { uunet!gary@sci34hub  }    System Mismanager
   SCI Technology, Inc.  OEM Products Department  (i.e., computers)
      Hestons' First Law: I qualify virtually everything I say.