Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!swrinde!elroy.jpl.nasa.gov!lll-winken!ncis.tis.llnl.gov!dog.ee.lbl.gov!elf.ee.lbl.gov!torek
From: torek@elf.ee.lbl.gov (Chris Torek)
Newsgroups: comp.unix.internals
Subject: Re: executing a stream / a compressed file
Message-ID: <9950@dog.ee.lbl.gov>
Date: 15 Feb 91 09:42:15 GMT
References: <1991Jan15.204849@IASTATE.EDU> <118868@uunet.UU.NET>
Reply-To: torek@elf.ee.lbl.gov (Chris Torek)
Organization: Lawrence Berkeley Laboratory, Berkeley
Lines: 35
X-Local-Date: Fri, 15 Feb 91 01:42:16 PST

>In <1991Jan15.204849@IASTATE.EDU> spam@IASTATE.EDU (Begley Michael L) writes:
>>What I'd like to do is execute a stream. [eg]
>>   uncompress -c microemacs|execute

In article <118868@uunet.UU.NET> rbj@uunet.UU.NET (Root Boy Jim) writes:
>I once suggested to Chris Torek that the kernel should execute
>compressed programs. He groaned.

Here is why:  A compressed file (that is, one which has had redundancy
removed by the compress(1) program) is a variable-length bit stream in
which the meaning of any given set of bits depends on an arbitrarily
large number of preceding bits (well, up to the compression table
sizes).  In other words, there is no way to tell that bit 71342 is the
start of a 9 bit long field whose meaning is `the sequence "hello"'
without starting at bit 0 and uncompressing until bit 71342 is reached.

There are forms of compression that can be uncompressed on a per-block
basis; these are somewhat more suitable.  There remains one stumbling
block: Unix systems like to run executables out of their original files
(at least in part), but the mapping between `original file blocks' and
`memory pages' is rather `lumpy'.  With regular files there is a direct
mapping in both directions.

This too could be changed, at some expense; alternatively, the kernel
could run compressed binaries by uncompressing them into a swap image,
and running the swap image (this would be easier in pure swapping
systems) or by uncompressing them into ordinary unlinked files and
running those.  The implementation of the latter is trivial even in
user code.  If one were to do this, however, the best approach would
probably be the one outlined by some other poster: run a nightly script
to compress executables that have not been read recently, and leave
executables uncompressed once they have been run.
-- 
In-Real-Life: Chris Torek, Lawrence Berkeley Lab EE div (+1 415 486 5427)
Berkeley, CA		Domain:	torek@ee.lbl.gov