Xref: utzoo comp.sys.amiga.datacomm:142 alt.flame:27706 Path: utzoo!utgpu!cs.utexas.edu!sun-barr!ames!vsi1!zorch!xanthian From: xanthian@zorch.SF-Bay.ORG (Kent Paul Dolan) Newsgroups: comp.sys.amiga.datacomm,alt.flame Subject: Re: A more memory efficient compress Message-ID: <1991Jan28.192438.14385@zorch.SF-Bay.ORG> Date: 28 Jan 91 19:24:38 GMT References: <20680@know.pws.bull.com> Followup-To: comp.sys.amiga.datacomm Organization: SF-Bay Public-Access Unix Lines: 54 ben@epmooch.UUCP (Rev. Ben A. Mesander) writes: > I've had another problem with compress. I run AmigaUUCP 1.06D, and > when I get a compressed tar archive, I normally unpack it with: > uncompress But with large archives, say bigger than 600K, this hangs. Right > before it hangs, I see some garbage graphics appear at random on my > screen, implying that a wild pointer has hit chip memory. Has anyone > else seen this? If it's less than about 1.5 megabytes, I can > uncompress the file, and then untar it in seperate steps. If it's more > than 2 megs, I have to transfer it to a unix system, unpack it, and > rearchive it with zoo. I have a 2.5MB Amiga 1000. See my parallel answer to yours for more detail, but one thing I did forget to mention there, that may explain your observation, is that the compress algorithm grows its look-back buffer dynamically, doing the equivalent of "-b9", then ..., then "-b12", then "-b13", then "-b14", etc. as it goes along. If a buffer grow doesn't work/isn't checked, then stuff could get overwritten. For small files, the look back buffer apparently doesn't get big enough to cause problems, for big (enough) ones it does. [My memory for the compress algorithm makes "vague" an understatement, but I think the buffer works with a compressed version of the data, rather than the raw bytes, so the point in an input file where it overflows and tries to double in size again may be input data dependent.] I'm a bit surprised it hasn't caused worse problems than you describe, though; a wild pointer is always a joy to behold. ;-) As I recommended in my other answer, _always_ doing an uncompress, and then a compress -b12, before downloading, should take care of your problem, since the "-b" flag controls the _maximum_ look-back buffer size that will be used to compress, and thus be required to uncompress, the data. Either because my "the look-back buffer" is really a family of buffers, or because of a lot of pointer overhead per data byte, I seem to remember that "-b12" corresponds to 64Kbytes of buffer space, with subsequent higher "-b" values doubling that. [Oh, yes; in another sense, the ## in "-b##" is the length in bits of the back-pointers being written to the output data, but that also corresponds to the distance==length of look-back table such a pointer can span, and thus the required table size that pointer supports.] Compression algorithms are fascinating; I could watch other people write them all day. Maybe some day I'll be well enough to resume work on mine. Kent, the man from xanth.