Path: utzoo!utstat!jarvis.csri.toronto.edu!mailrus!uflorida!novavax!twwells!bill
From: bill@twwells.com (T. William Wells)
Newsgroups: news.software.b
Subject: Re: CNews building up files in "in.coming"
Message-ID: <1989Nov13.101830.19896@twwells.com>
Date: 13 Nov 89 10:18:30 GMT
References: <1989Oct22.192159.4827@acd4.UUCP> <1989Oct30.121734.1658@twwells.com> <1989Nov1.184509.27953@utzoo.uucp>
Organization: None, Ft. Lauderdale, FL
Lines: 60

In article <1989Nov1.184509.27953@utzoo.uucp> henry@utzoo.uucp (Henry Spencer) writes:
: In article <1989Oct30.121734.1658@twwells.com> bill@twwells.com (T. William Wells) writes:
: >I typically have about 10M free on my disk just before expire
: >runs; unfortunately, I also tend to run jobs that do a lot of
: >data manipulation. One core dump, or one extraneous big data
: >file, can make that 10M disappear real quick!
:
: Unfortunately, this is really hard to deal with in any graceful way.  Even
: with a suitable change to the strategy in newsrun, there is still a problem
: with C News's basic approach, which is to anticipate space problems rather
: than trying to cope with them when they arrive.  There is an inherent
: assumption that the free space at a given time is a reasonable prediction
: of what the free space will be in the immediate future.
:
: (C News does make some effort to cope well with running out of space.
: Unfortunately, it is impossible to really do this right.  There are too
: many situations where space exhaustion in mid-stride means you trip and
: fall.  All the more so when stdio, dbm, the shell, etc. are involved as
: middlemen and you don't have direct access to the problem.  Hence the
: emphasis on prevention rather than cure.)

A generally good idea. The thoughts I have are based on a
different kind of prevention: minimize the amount of extraneous
space used by C news, especially the transient space, so that the
probability of running out of space is diminished.

So far I've been lucky. The closest I've come is a runaway
process. I caught it with just 20K to spare....

: >What I'd like to see, as a configuration option is:
: >
: >"Your system may be one of those where the amount of free disk
: >varies outrageously and unpredictably. If so, or if you have
: >other reasons, would you like C news to minimize disk usage,
: >almost certainly at the cost of increased processing time? [n]"
:
: I conjecture that this is a relatively rare situation.  If there are lots
: of people with such problems, I'd like to hear about them.  (Note, we are
: talking about major variations over very short time periods:  minutes not
: hours.)  I fear it would not be trivial to fit this in; I'm reluctant
: to do it unless there is widespread need.

Well, did you get much of a response?

BTW, the things I was thinking of doing, in response to a yes to
the above, were stuff like deleting each input file after it was
successfully processed and compressing the backup log and history
files. Other things, like controlling batching and the length of
time news is stored are already configurable. This would probably
save almost 2M on my system during peak usage, a significant
savings indeed (recall: at expire time I have 10M free. Actually,
no longer true: now 9M.)

Of course, I can patch C news to do all this myself, but if
others have the same problem, having it as a configuration option
would be nice.

---
Bill                    { uunet | novavax | ankh | sunvice } !twwells!bill
bill@twwells.com