Path: utzoo!censor!geac!torsqnt!lethe!yunexus!ists!helios.physics.utoronto.ca!news-server.csri.toronto.edu!cs.utexas.edu!rice!uupsi!njin!limonce From: limonce@pilot.njin.net (Tom Limoncelli) Newsgroups: comp.sys.amiga.datacomm Subject: Re: A more memory efficient compress Message-ID: Date: 29 Jan 91 06:18:13 GMT References: <20680@know.pws.bull.com> <1991Jan28.192438.14385@zorch.SF-Bay.ORG> Organization: Drew University - Madison NJ Lines: 28 In article <1991Jan28.192438.14385@zorch.SF-Bay.ORG> xanthian@zorch.SF-Bay.ORG (Kent Paul Dolan) writes: > Compression algorithms are fascinating; I could watch other people > write them all day. Maybe some day I'll be well enough to resume > work on mine. I was thinking the other day, it's possible to beat LZ compression by an order of magnitude by methods that just aren't as fast. LZ trades size for super speed. A good tokenizing compression could be a killer for Usenet news, but you'd need a new token table for each class of newsgroup (sex, social, technical, etc.). I wonder if you could do a statistical analysis of each newsgroup, figure out which ones have similar tables, and compress those using their tables, sources and binary newsgroups using LZ compress, etc. In fact, the token tables could be dynamic. We could create something like comp.mail.maps (news.compress.tables?) which would constantly be posting the tables generated for last months newsflow. I've got to stop wrting email late at night. This almost makes sense. Tom -- One thousand, one hundred, seventy five people died of AIDS last week. Did someone mention a war in Iraq?