Xref: utzoo news.software.b:1519 news.software.nntp:59 Path: utzoo!attcan!uunet!yale!husc6!ukma!david From: david@ms.uky.edu (David Herron -- One of the vertebrae) Newsgroups: news.software.b,news.software.nntp Subject: Re: SPOOLNEWS & nntp Message-ID: <10118@e.ms.uky.edu> Date: 3 Aug 88 03:50:07 GMT References: <10080@e.ms.uky.edu> <14956@oddjob.UChicago.EDU> Reply-To: david@ms.uky.edu (David Herron -- One of the vertebrae) Organization: U of Kentucky, Mathematical Sciences Lines: 83 In article <14956@oddjob.UChicago.EDU> matt@oddjob.UChicago.EDU (Maxwell House Daemon) writes: >David Herron sez: >> It's amazing what you'll do when you're bored. One afternoon I moved the >> SPOOLNEWS code from inews.c into nntpd to see what sort of effect it would >> have on the system load, it "seems" to help quite a bit. >This may be a big loser, David. I know. I knew that when I posted it. >Until you process the article it >won't be in the history file. I know. I've been running with SPOOLNEWS all along. That's because I wanted to be able to control how many "streams" of processes were unbatching news. The cost is two fork()/exec() executions per article to process the article. The first set is to save away the standard input into SPOOLDIR/.rnews and the other is to examine it to determine if it is already present in HISTFILE and if it is not to insert it into the right place in the news hierarchy. On a normal system however you can easily get multiple streams of rnews processes running, if you have SPOOLNEWS undefined. That will happen whenever you have >1 UUCP neighbor shipping you news at once. Or if you have multiple NNTP neighbors. With multiple streams of these processes running our uVaxII gets veery sloooowwww. And I have a vested interest in this machine not getting bogged down since it's the one where I 'live' :-). >Hence nntpd will continue to accept >more copies of that article until you run rnews -U. This increases >the network load, which defeats one of the goals of NNTP. Yes I understand that. I've switched from running my news scripts every 15 minutes to every 10 minutes, plus I've staggered the news transmission scripts with the news reception scripts so that they will often sidestep each other. There is also flock stuff going on so that not more than one "newsdaemon" script is running on a machine at any one time. If 10 minutes isn't fast enough a turn around time I could decrease the granularity to 5 minutes or some such. It's merely a matter of editting crontab ... There is a trade-off between network load and host load. I have more host load than I can handle, but with a reshuffling of what happens when I can handle it. As I see it the better fix is one which was discussed on nntp-managers some time ago (but not yet implemented to my knowledge). That is to put something into the nntp_access file which will put limits onto the number of accesses from certain subsets of the network. This could be on a per-host, per-net or anybody else basis. If I could use that to force only (say) 2 connections from the outside world, then this machine could handle the load. I would be able to turn SPOOLNEWS off in nntp, and possibly in news in general. This is yet another version of the local policy decisions versus global policy decisions debate. In my case I have a colleague who was very adamant that I do something about nntp and the load it causes. This was the easiest fix. My patch does provide more flexibility in nntp administration ... It also saves a fork()/exec() pair if you have SPOOLNEWS defined in the underlying news system. (the one which writes the article into SPOOLDIR/.rnews). >Since you >say you often have multiple incoming NNTP sessions, I think you will >get multiple copies quite often. Check your log. I know that oddjob >sometimes get offered the same article from two sources (separated by >over 1000 miles) mere seconds apart. Unfortunately something broke the syslog stuff in news here a couple of months ago and I haven't had time to fix it... sigh. I've answered in length -- probably greater length than most of us need -- in order to let Matt know that I know what I'm talking about. Also if *is* a flaw in my reasoning then someone can point it out and let me correct my ways. -- <---- David Herron -- The E-Mail guy <---- ska: David le casse\*' {rutgers,uunet}!ukma!david, david@UKMA.BITNET <---- <---- Looking forward to a particularly blatant, talkative and period bikini ...