Newsgroups: news.software.b
Path: utzoo!henry
From: henry@zoo.toronto.edu (Henry Spencer)
Subject: Re: expire oink
Message-ID: <1990Aug31.173105.13426@zoo.toronto.edu>
Organization: U of Toronto Zoology
References: <1990Aug22.165739.6918@lethe.uucp> <49312@olivea.atc.olivetti.com> <1990Aug30.162009.2017@zoo.toronto.edu> <49326@olivea.atc.olivetti.com>
Date: Fri, 31 Aug 90 17:31:05 GMT

In article <49326@olivea.atc.olivetti.com> jerry@olivey.olivetti.com (Jerry Aguirre) writes:
>The dbz file looks like:
>
>	dbz 3 999983 9 = 128 127 24 4 0 1 2 3
>	265832 0 0 0 0 0 0 0 0 0 0
>
>From which I calculate a 4 Meg size using Henry's formula.  That is for
>a history file with 269616 lines...

"Houston, we've got a problem."  Something is very wrong here.  For one
thing, all those zeros in the second line are a bad sign:  there is some
problem with expire, by the looks of it.  The second line is basically a
history of recent entry counts, with the first number being current count.
Those zeros say that the database is being rebuilt from scratch rather
than using the old one as a basis, which is what C expire now tries to do.

For another thing, your table is way too big.  For 269k entries, a table
of about 350k slots is lots.  Dbz 3 handles overflows gracefully; there
is no need to grossly over-size the table to avoid risk of overflow.
If you can find out why expire isn't using the old database as a basis
for the new one, this problem will solve itself, since expire will re-size
the table based on the actual usage.  (That's what the usage history is
for...)  The re-sizing calculation is based on piles of graphs from
simulations of performance tradeoffs, by the way, not just guesswork.
Whole trees died for dbz 3...

In any case, to get, say, a 10MB table, which is about what you'd need
to push expire up to 11MB in the absence of bugs, you would need to be
keeping about 18 months of history.  I doubt that anyone is doing that!
-- 
TCP/IP: handling tomorrow's loads today| Henry Spencer at U of Toronto Zoology
OSI: handling yesterday's loads someday|  henry@zoo.toronto.edu   utzoo!henry