Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sun-barr!newstop!sun!amdcad!dgcad!proa.sv.dg.com!gary From: gary@proa.sv.dg.com (Gary Bridgewater) Newsgroups: news.software.b Subject: Re: Steady state news. Message-ID: <1990Dec12.093657.1488@proa.sv.dg.com> Date: 12 Dec 90 09:36:57 GMT References: <1990Dec11.231124.24426@zoo.toronto.edu> Organization: Data General SDD, Sunnyvale, CA Lines: 52 I really like this idea! In article <1990Dec11.231124.24426@zoo.toronto.edu> henry@zoo.toronto.edu (Henry Spencer) writes: >In article flee@dictionopolis.cs.psu.edu (Felix Lee) writes: >>This is an extreme form of spacebased expiry and inherits all of its >>control problems. Deciding which articles to expire may be difficult. Keep a FIFO file per expire rule - i.e. a 1day file, 2day file, 1week file. Maintaining the files is a pain but you could keep a separate tell() index to the current actual beginning and only rebuild rebuild from time to time. New articles just go on the end via append. This assumes that the current history file is split in two - one file to hold the IDs and another (set of) file(s) that contains the posted/expire/article# info. It might be handy to keep the size as well. This data is all jammed together now since it is processed at the same time. The problem is or has been coordinating them - i.e. knowing when to drop articles and when IDs can be let go. But I don't think they need to be coordinated much beyond keeping an ID _at least_ as long as you keep the article. It should be possible to devise a method to drop IDs separatly from the date - perhaps an 8 bit pseudo-time stamp. That is, e.g., this is week 0 so we scan the database and drop all old week 1 values, next week is week 1 and we scan and drop all week 2 values. After week 0xff we just wrap. I don't know where the 8 bits are going to come from - maybe 4 or 6 is enough. It's just a thought. >>Has anyone figured out how spacebased expiry should work? This should be answered on a per site basis. You know where you store news and how to check the current utilization and limits. The software would invoke your function and could either ask "How much should we dump?" or "Should we dump more?" or some simple question for which the answer can be determined. When you have built the module you "compile" the expire system - i.e. run the script producing script. >We thought about it a bit for C News, and concluded that the policy issues >are complicated (it's simple if you expire everything at the same time, >but the interactions with selective expiry get messy) and we didn't feel >like solving them. Don't apologize for it. Breaking up the functionality of posting and expiring was a _Good_ _Thing_. Thanks. It opens up a lot of possibilities and lets effort be spent on individiual components without destroying your whole news system. Hacking B-news expire can be very scary. Granted that adding all this file manipulation is a pain - dealing with 15 Mbytes a day is already a pain and it isn't going to get better. -- Gary Bridgewater, Data General Corporation, Sunnyvale California gary@sv.dg.com or {amdahl,aeras,amdcad}!dgcad!gary C++ - it's the right thing to do.