Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Posting-Version: version B 2.10.1 6/24/83; site clyde.UUCP Path: utzoo!linus!decvax!harpo!eagle!mhuxl!ulysses!cbosgd!ihnp4!clyde!lda From: lda@clyde.UUCP Newsgroups: net.news.adm Subject: how batch & unbatch work Message-ID: <224@clyde.UUCP> Date: Tue, 25-Oct-83 13:09:39 EDT Article-I.D.: clyde.224 Posted: Tue Oct 25 13:09:39 1983 Date-Received: Wed, 26-Oct-83 07:51:44 EDT Organization: Bell Labs Whippany NJ Lines: 135 There is a news batching system that is distributed with news 2.10.1 that makes transmission of news much more efficient. News that is batched takes much less time to transfer. This is true because uucp has to process only three files per batch rather than three files per article. Another side effect is that the size of the spool directory is kept smaller, meaning that uucp has less looking to do to find each file. This system is very simple, yet very effective. The purpose of this article is to encourage netnews administrators to use a batching scheme in their news transmission, and to show how easily news batching can be set up at a site where news 2.10.1 has been installed. For the purposes of this discussion, I have assumed that the following definitions were made in defs.h during the make of news 2.10.1 (Note: this is the way that the news source was distributed): #define BATCH "/usr/lib/news/unbatch" /* name of unbatcher */ #define BATCHDIR "/usr/spool/batch" /* location (dir) of batching files */ There are four programs involved in the batching scheme that is distributed with news 2.10.1. They are uux, rnews, batch and unbatch. When news comes in from a system, rnews "posts" it on the local system and then decides which systems are to have it forwarded to them. If the entry for a remote system (eg. "rmt") has ":F:" in the third field, then the fourth field is taken to be a filename into which the location of the article (on "local") is written. For example: The sys file for system "local" looks like this: local:net,fa,nj:: rmt:net,fa,nj:F:/usr/spool/batch/rmt If an article is posted on local to net.news.b and it becomes article 909 in that group, a line will be written into the file "/usr/spool/batch/rmt" that says /usr/spool/news/net/news/b/909 Batch is a program that reads the list of filenames that has been created by rnews and makes a "batch" of news and writes it on standard output. This "batch" is just a concatenation of the articles named in the list separated by a line that says: #! rnews length where "length" is the length of that article in bytes (characters). Once all the articles have been written to stdout, the list of filenames is truncated to make ready for the next batch. If the file "/usr/spool/batch/rmt" had the lines /usr/spool/news/net/news/b/909 /usr/spool/news/net/jokes/1222 /usr/spool/news/fa/telecom/401 and you executed the command /usr/lib/news/batch /usr/spool/batch/rmt then the following batch would be written on the standard output and /usr/spool/batch/rmt would be made empty. #! rnews 1234 #! rnews 6833 #! rnews 4563 "What good is all this?" you ask. "Why would I want to write stuff to stdout and loose it?" Good question. This is where uux comes in. Uux is a program that lets you execute a program on a remote system, "rmt". Rather than letting the output of batch go to stdout and be lost (cf. the story of Onan in the Bible), you pipe it into the stdin of uux and send it as the input to the rnews command command on "rmt". This is how that's done: /usr/lib/news/batch /usr/spool/batch/rmt | uux - -r -go -z -n rmt!rnews The flags are interpreted to mean this: - read standard input as input for rnews. -r queue but don't send right away. -gN give the job grade N. I use N=o to give news batches a grade that's lower than regular uucp jobs. Mail goes out as grade A. -z don't notify user on zero exit status of rnews. -n don't notify me period. SO, now you have a batched job queued for the remote system. When "rmt" receives the batch and executes rnews on it, "rmt"'s rnews will see the "#! rnews 1234" on the first line and say to itself, "Aha! This is a batched job. I know what to do now." It will then call "unbatch" (the last link in the chain) to break the articles apart and post them as it sees fit. To make all this work together well, you should run batch from cron periodically, say every hour (more often on backbone sites). This can be done for all the systems with which you exchange batched news by using a shell file that contains command lines for each of those systems and then invoking the shell from cron. Be sure that the directory "/usr/spool/batch" exists and is writtable by the rnews program. There is a script that is distributed with news 2.10.1 called sendbatchednews that will do what you want, but it requires some modification to uucp. This is the script I use: # @(#) /usr/lib/news/batch.sh 1.1 for rmt in akgua burl floyd ihnp4 masscomp do # don't create a batch if the batch list is empty if test -s /usr/spool/batch/$rmt then /usr/lib/news/batch /usr/spool/batch/$rmt | uux - -r -go -z -n $rmt!rnews fi done /usr/lib/uucp/uucico -r1& # send out the queued news This is our sys file: clyde:: akgua::F:/usr/spool/batch/akgua burl::F:/usr/spool/batch/burl floyd::F:/usr/spool/batch/floyd ihnp4::F:/usr/spool/batch/ihnp4 masscomp::F:/usr/spool/batch/masscomp # make a list of everything that comes in # be sure to 'mv list list.' each night list:all:F:/usr/spool/batch/list -- Larry Auton WECo @ BTL WH 2C-123 (201)386-4272 ihnp4!clyde!lda