Path: utzoo!utgpu!watserv1!watmath!att!linac!pacific.mps.ohio-state.edu!zaphod.mps.ohio-state.edu!swrinde!elroy.jpl.nasa.gov!ncar!asuvax!anasaz!rusty
From: rusty@anasaz.UUCP (Rusty Carruth)
Newsgroups: news.software.b
Subject: expire
Summary: one reason why there are so many hacks around it (IMHO, of course!)
Message-ID: <4313@anasaz.UUCP>
Date: 19 Jun 91 22:29:16 GMT
Reply-To: rusty@anasaz.UUCP (Rusty Carruth)
Organization: Anasazi, Inc.
Lines: 93

Some time back someone remarked about the glut of "expire" replacements,
and how that they felt that people were going astray by not using the
"supplied" expire program.  (massivly interpreted/filtered observation
there - please note that this was/is NOT intended as a flame, nor am I
intending to be casting aspersions on ANYBODY's work!  I am VERY thankful
for all the software others have supplied which supports news!)

I've finally come to the point that I am able to make a semi-rational
statement as to why I believe that this happens.

When I run expire, I am usually wanting to free up some given amount
of disk space (for incoming news to land in, for example).  I rarely
am thinking of limiting the length of time articles hang around in
a newsgroup - I simply want to get some free disk space without
angering my users too much.

So, when I "write" an expire table, what do I enter?  Retention times.
and, if I don't get enough space from the expire, I either have to
make the times shorter (and remember to go back and change it after
expire has finished), manually remove some stuff, or ...

I currently administer 2 news machines.  One is in the process of
being converted to cnews, the other is still on Bnews (the one I'm
posting from, as it happens).  I've been using "reap", with some mods,
here on "this machine" (anasaz) for some time with pretty good results,
except that I've got a VERY complex reap list, which makes adding new
news groups a pain.

The algorithm I'm hoping to implement runs something like this:

  Set a goal for the amount of free space you want now.
  lump newsgroups into one of 3 categories: junk, good, archive (archive
      is not currently being done)
  set "high" and "low" limits for each newsgroup (see below for their use)
  set "rate" values for each newsgroup (also see below)

  for each junk group, expire anything older than 1 day and see if you have
  freed up the space desired if so stop, if not, continue to:

  for each good newsgroup:
  
     expire any articles older than the "high" limit for this newsgroup

     if avail space > desired space, stop

  end for

  if we still need more space:

  for each newsgroup  (note - junk groups included)

    if oldest article is older than "low limit" then

      expire "rate" days of articles (i.e. if rate = 1, expire 1 days worth)

    if avail space > desired space, stop

  end for

Another person here has an idea based upon priorities and such, but it seemed
even harder to implement than my hare-brained idea :-)

Reading the doc for expire, it looks like I could add another field to the
middle field of the history line which contains the SIZE of the file (thus
saving me from having to scan the entire directory structure to calculate
file sizes).  Would there be any massive problems with doing this?

(Note that my intention is to run the above algorithm from top to bottom,
THEN actually remove the files, thus allowing me to traverse the tree only
once)

Also, I take it that a '-' in the second subfield means that the
article has been expired?

Anybody crazy enough to help me on this insane project?

Would the "powers-that-be" be interested in including my version of
"param_expire" (or whatever in the world it turns out to be called)
in future Cnews's (as an optional method for expiration)?  (Assuming
that I get it finished this century...)

Is this even a good idea, in other folks' minds?  (PLEASE, if you reply
to this question, notice and address the issue of <why it is we run expire 
in the first place>  (see paragraph 3 above)).

"Raving wildly, Rusty hits the "s" key in rn"  :-)

Rusty


{ames!ncar!noao!asuvax,mcdphx}!anasaz!rusty      anasaz!rusty\ 
73 de Rusty Carruth, N7IKQ  (602) 870-3330   anasaz.UUCP!rusty>@asuvax
P.O. Box 27001, Tempe, AZ 85285             rusty%anasaz.UUCP/   \.eas.asu.edu