Path: utzoo!attcan!uunet!cbmvax!snark!eric From: eric@snark.UUCP (EricS.Raymond) Newsgroups: news.admin Subject: how to lower news-transmission overhead Message-ID: Date: 13 Dec 88 17:26:47 GMT References: <26469@bu-cs.bu.edu> <267@ssbn.wlk.com> <486@mccc.uucp> Organization: Erisian Liberation Front Lines: 43 Back-References: <2013@van-bc.uucp> In article Karl Kleinpaste writes: I believe Karl is correct, and have done something about it. Henry Spencer and I discussed this at the '86 Worldcon. He told me at the time that his profiles showed ngmatch() eating up 15% or more of the elapsed time in rnews runs, and passed along a nifty idea for reducing the bite that he said was going to be implemented in C news. I used it. If you compile 3.0 with FEEDBITS on, per-newsgroup subscription bits are computed en masse *once* at the start of each rnews run. If you compile with CACHEBITS, this information is stashed away in half-compiled form the first time it's computed, and the high-overhead part of the computation is only done when the sys-file-equivalent changes. Result? On most runs you pay only for N sscanf(buf, "%x", ....) calls, where N is the number of currently active groups. This is a significant win. Another helpful feature I've implemented for shortening subscription lines is a {foo,bar,baz,...} alternation syntax like that of csh. My ngmatch() does the right recursive tricks so that combinations of ! and {...} work and nesting of alternations works. Time overhead for this new feature is insignificant, and it allows subscriptions to be expressed much more concisely. Consider comp.sys.{all,!ibm.pc,ibm.pc.rt,!mac,!amiga,!next,!sgi,!sun} This is exactly equivalent to: comp.sys.all,!comp.sys.ibm.pc,comp.sys.ibm.pc.rt,!comp.sys.mac, !comp.sys.amiga,!comp.sys.next,!comp.sys.sgi,!comp.sys.sun Which would *you* rather read? :-). And because computation cost for ngmatch() is mostly a function of subscription list length, the top version resolves more quickly. Finally, I should note that Shane McCarron's UUCP multicast code is also incorporated in 3.0. Yes, somebody *has* been listening...and I will continue to put a high priority on supporting features that reduce and help control the costs of feeding other sites. -- Eric S. Raymond (the mad mastermind of TMN-Netnews) Email: eric@snark.uu.net CompuServe: [72037,2306] Post: 22 S. Warren Avenue, Malvern, PA 19355 Phone: (215)-296-5718