Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!think.com!sdd.hp.com!wuarchive!uunet!mcsun!hp4nl!svin02!eba!ebh.eb.ele.tue.nl!wjw From: wjw@ebh.eb.ele.tue.nl (Willem Jan Withagen) Newsgroups: comp.sys.apollo Subject: Re: Info on setting up net news.(Very Long and might be drairy) Message-ID: <1207@eba.eb.ele.tue.nl> Date: 10 Jun 91 11:54:51 GMT References: <9106072205.AA12094@hobbes.mdc.com> Sender: news@eb.ele.tue.nl (The News system) Reply-To: wjw@eb.ele.tue.nl Organization: Digital Systems, Eindhoven University of Technology, the Netherlands Lines: 183 In article <9106072205.AA12094@hobbes.mdc.com>, johnb@HOBBES.MDC.COM (John Breen) writes: => >Could some kind soul point me to the latest versions of the sources [of news => >software] I will need and point out any configuration problems I should be => >aware of? => => Ditto. => => Actually, I've been researching this a bit lately, so I'll throw out what I've => found and see if anybody has any corrections/additions. => => For an internet (read non-UUCP ?) site, I've been told (by a news administrator => that I'm going to try to get a feed from) that I would need the following (we're => connected to the news "source" via NSFNet): => " - cnews => - nntp => - rn, nn, or trn => - plenty of disk space..." => I'll try to make a clean as possible reply to this question as to what I know about running news, especially on apollo's. (This is for shure not complete!!!!! ) First the're two different parts to news: 1) the newsfeed side, which involves the retrieval of the articles. 2) the user side, where the reading is done 1) The news system revolves around the news-article database, which consists of at least the following files: active The system keeps a file ( ~news/active ) in which it keeps the article numbers of the available articles. spool The tree where all articles are kept. Note: an article is just a file in the news-spool tree, where the groups are translated into directories. history Which is a dbm-files for fast retrieval of certain info. sys tells what systems can feed news, and what groups, what systems are to be loaded with news. (there are more but these are the most essential.) Then there's the program inews which is the maintainer of this database. It excepts articles from other hosts, and if they comply puts them in the database. If any systems get a feed from this system, they might be forwarded by inews. (NOTE: might) The problem on Apollo's lies in the fact that inews wants to be the only one writting this database. And that includes inews on other systems. MUltiple access is not counted for. If another workstation has a read access on this file, the inews will fail updating the database, creating possible problems. Who the newsfeed is arranged depends on the connecntion type of the systems: - There might be UUCP. Then batches of newsarticles are created and these batches get transfer on regular basis by an apropriate program. - If there's a faster link (ethernet,.. ) the this still could work this way. Only the program which does the actual transfer is different. In both case the batches are setup by inews and subprograms. - Then there's the nntp protocol, which runs on TCP/IP, and allows for faster access. The nntp-daemon can look into the database and supply the client with all kinds of info. It can also accept new articles which then get inserted into the database by inews. nntp clients could be other systems which require a newsfeed, but also certain types of newsreaders (see 2) ) This NNTP protocol can be used in two ways: 1) The upstream newsfeed uses the protocol to download news into a downstream system. Compare this with the batches. The client and the server first negotiate which articles are new, after which the new ones are downloaded. (A possible program for this could be nntpxmit) 2) The reverse aproach is also possible: The downstream site polls the upstream site for new articles. The interaction is more or less the same. (A possible program for this is nntpxfer) (I know most about the nntp-stuff, since that is what we use.) The question is where do terms like {B,C}-news fit in. {B,C}-news implement the nntp-daemon and the programs to do the database management. Supposedly is C-news more effective. (I can't tell since we only run B-news) The last item to discuss is the generation on articles. As long as they are local, there's no problem. They are insterted into the database by inews at that's it. If however the article has a larger distibution then it needs to be transfered to other systems. For this a backbone has been created, once you get the message at a backbone site, it'll get distributed to all apropriate systems. The configuration of what to do with 'home'-generated articles is described in the 'sys'-file. It specifies what articles go where. The transport is again taken care of by inews. Inews can diffentiate between downloaded and uploaded articles by looking at the path in the article. Any of the hosts in the sysfile not in the path, get a 'copy' of the article processed by inews. The sysfile entry also supplies the program which has to do the transfer. This could be a queueing program: the article gets queued, and once there's enough data, or time has expired, then the lot is transmited. (UUCP or NNTP or others. ) It could also be put in a mail message, and mailed to a system which has a special program for receiving news-articles (connected to an account name). This special program calls upon inews to actually do the news processing. So it again gets distributed along the entries in the sysfile. (One could use this method to download news as regular feed, but it gives a tremendous overhead in mail-messages) At our site is the local ammount of articles generated very low, so we use the last method to upload 'not-local' news to our Univeristy central news system. What happens there, I don't know, and I do not have to know, since the connections to the outerworld are not in the path of the uploaded articles. And hence they get transfered to our national news system. (I guess) From there they'll go across the big pool, where the worlds other users get it. INTER: I've once got a responce within the hour form somebody in California. APOLLO: The apollo problems arise from the fact that if node A is the site where somebody is reading the database (mainly the active,history-files) then running inews on node B causes lots of trouble because of the implementation of the file system. (On SUN/ea. this is solved in a dirty way by NFS ) The solution is to run all actions on the database on one system. So only one node really runs inews. On other systems there's a fake inews which forwards the inews-stuff to the real inews. (This requires running NNTP) This still leaves us the trouble that newsreader might require the active file for reading too. For this I've created active.copy which gets updated from the actual active file every time inews updates the active file. The newsreaders are then patched to use the copy, which can be read by all. If inews wants to copy then it fails on the 'copy', but the original is still in good order. The problem lies in collecting all the required items. The hacking is only a matter of small changes to pathnames in config files. 2) Let's assume that we already have a news system up and runing. Then the user has a newsreader program, which uses a ~/.config-file to keep a log of what the user has read up to now. The news system has it's active file. Reading both gives the newsreader the info of what articles to represent to the user. Now there's two category of nesreaders: (certainly not complete) Those that use the news-database directly: - rn, vnews, nn, readnews, .... Those that go through a NNTP server. - rrn, xrn, ...... The first require that care is taken of the problems with inews and access to the database. The second require a NNTP-sever on a per active user. APOLLO: The second one has two disadvantages. 1) It requires an up and running deamon per user reading news. And before 10.3 this was a serious problem on systems running as gateways, which would usually also be news-gateway. 2) All transfers go through a deamon and TCP/IP which is not the faster way of accessing things. The advantage is that all the available programs can be run as found on the net. They require very little hacking. The first aproach requires slight modifications to the readers and the scripts with the readers. But the speed is a LOT more impressive, and the burden on the gateway is negliable. [ rn however runs in raw pad/vt100 mode, which is not flawless as we all know my rn alias is stty -cbreak; \ /usr/local/bin/rn -F'''=> ''' -g5 -H -S -hMessage-ID -hDate ; \ stty -cbreak which seems to take care of most problems. ] Hope it's usefull info to someone, Ciao, Willem Jan -- Eindhoven University of Technology DomainName: wjw@eb.ele.tue.nl Digital Systems Group, Room EH 10.10 P.O. 513 Tel: +31-40-473401 5600 MB Eindhoven The Netherlands