Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!seismo!brl-adm!adm!JOSH@ibm.COM
From: JOSH@ibm.COM (Joshua W. Knight)
Newsgroups: comp.unix.wizards
Subject: Re:  Filtering Everything
Message-ID: <6946@brl-adm.ARPA>
Date: Fri, 17-Apr-87 00:28:38 EST
Article-I.D.: brl-adm.6946
Posted: Fri Apr 17 00:28:38 1987
Date-Received: Sun, 19-Apr-87 10:12:39 EST
Sender: news@brl-adm.ARPA
Lines: 41

In response to earlier remarks by Jim Cottrell, Stefan Vorkoetter
(smvorkoetter%watmum.waterloo.edu) asks

 >
 > What do you mean, the program would have to live in my terminal?  If you
 > mean that my terminal would have to know about the data compression, then
 > that is no problem.  I am not using a terminal; I am using a terminal
 > emulator which my company (S. M. Vorkoetter Software) markets, and to which
 > I obviously have the source code.  My idea is to compress data going from
 > the host, and uncompress it when it reaches the emulator, a process that is
 > completely transparent to the user of the system.

To which Jim Cottrell replies:

 >
 > Except that in order to compress data, you need to collect statistics on
 > frequency use. So you need to synchronize the xmt'er and rcv'er tables,
 > unless you periodically transmit them. Also,
 > suppose you *can* compress the data, say, four input bytes down to three.
 > What do you do when the third character is a line feed, or if you are
 > using character I/O for input? You lose your advantage, and you force
 > some means of deciding whether to buffer and encode or send it now,
 > because the data is needed.
 >
 > It's *NOT* hopeless, but it's not trivial either. Telnet protocol
 > provides `macros' as well as other features. Other protocols provide
 > abbreviations for repeated characters. Have fun in any case.
 >
 >     (Root Boy) Jim "Just Say Yes" Cottrell    <rbj@icst-cmr.arpa>
 >     I want to kill everyone here with a cute colorful Hydrogen Bomb!!
 >

Internal to IBM there is a terminal emulation program that does exactly
what Stefan asks about, but for VM/CMS.  It uses a modification of the
Lempel-Ziv compression algorithm which constantly updates the "tables"
(actually a cache like algorithm).  On normal english text, this method
commonly gets a factor of 2 compression.

			Josh Knight
			IBM T.J. Watson Research Center
josh@ibm.com, josh@yktvmh.BITNET