Path: utzoo!attcan!uunet!lll-winken!ncis.llnl.gov!helios.ee.lbl.gov!pasteur!ucbvax!agate!labrea!rutgers!att!skep2!wcs
From: wcs@skep2.ATT.COM (Bill.Stewart.[ho95c])
Newsgroups: comp.binaries.ibm.pc.d
Subject: Re: Is uncompression faster than disk I/O?
Message-ID: <421@skep2.ATT.COM>
Date: 24 Jan 89 00:44:31 GMT
References: <14227@princeton.Princeton.EDU> <929@novavax.UUCP> <2863@stiatl.UUCP>
Reply-To: wcs@skep2.UUCP (46323-Bill.Stewart.[ho95c],2G218,x0705,)
Organization: AT&T Bell Labs Center 4632, Holmdel, NJ
Lines: 39

In article <2863@stiatl.UUCP> john@stiatl.UUCP (John DeArmond) writes:
} >> Someone suggested to me that it might pay off to store my data files
} >> in compressed format, then uncompress them when I get ready to use
} I've found that on my Compaq 386, it is MUCH faster to decompress
} on the fly than to read the raw data.  The CPU is so much faster than
} the I/O system that there is really no contest.  ........
} So Norman, the answer to your question is - IT Depends!  The
	That's really the correct answer.  With the more
	sophisticated algorithms, compression is often slow,
	but decompress is table-driven and very fast - if
	you are going to generate a data-set that you read a
	lot but seldom write to, and don't need to access
	randomly from the disk, storing it compressed can be
	a big win (assuming you've got enough memory around.)
	If you're going to do a lot of both, try LZW, but
	also try simple run-length encoding (RLE) for images.

} general purpose algorithms such as LZW (ARC, compress, etc) and
} Huffman do a pretty good job in the general sense but you can do
} much better if you can exploit some characteristic of your data
} set.  KISS principles apply fully here.  Many times a very 
} simple algorithm will achieve a high fraction of more complicated
} routines but with a vastly smaller implementation and execution
} time.  So, take your compiler in one hand and an editor in the 
} other and EXPERIMENT!

	I've seldom found anything to beat LZW, even when I
	know a *lot* about the special characteristics of
	the data.  For some data (e.g. images with lots of 
	empty space) a simple RLE can be worthwhile - 
	a small fast 90% solution may be better than a big, slow
	98% one, especially on a PC.  But document your
	algorithms *very thoroughly*, and don't waste too
	much time tweaking.
-- 
#				Thanks;
# Bill Stewart, AT&T Bell Labs 2G218 Holmdel NJ 201-949-0705 ho95c.att.com!wcs
#
#	News.  Don't talk to me about News.