Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!husc6!rutgers!sri-spam!mordor!lll-lcc!ames!oliveb!intelca!mipos3!cpocd2!howard
From: howard@cpocd2.UUCP (Howard A. Landman)
Newsgroups: comp.unix.wizards,comp.arch
Subject: Re: Large programs
Message-ID: <918@cpocd2.UUCP>
Date: Fri, 16-Oct-87 19:30:41 EDT
Article-I.D.: cpocd2.918
Posted: Fri Oct 16 19:30:41 1987
Date-Received: Wed, 21-Oct-87 20:33:24 EDT
References: <1046@ius1.cs.cmu.edu> <1130@gilsys.UUCP> <2946@sol.ARPA>
Reply-To: howard@cpocd2.UUCP (Howard A. Landman)
Distribution: na
Organization: Intel Corp. ASIC Systems Organization, Chandler AZ
Lines: 36
Keywords: UNIX LS HUMOR
Xref: mnetor comp.unix.wizards:5010 comp.arch:2679

In article <2946@sol.ARPA> crowl@cs.rochester.edu (Lawrence Crowl) writes:
>In article <1130@gilsys.UUCP> mc68020@gilsys.UUCP (Thomas J Keller) writes:
>]   2)  speaking of speed, we all seem to have forgotten that each one of those
>]lovely small programs in the chain has to be loaded from disk.  Clearly, the
>]overhead necessary to fork & spawn multiple processes, which in turn load
>]multiple program text into memory, is **MUCH** greater than spawning and 
>]loading a single program!  Waiting time is important too, you know?
>
>You forgot an important speed difference.  In the pipe approach, each program
>in the pipe does a lot of file I/O and string to data to string conversions.

???  A pipe need not do any file I/O at all!  The data is buffered in memory.
One of the advantages of pipes is that they still work when your file system
is full, whereas writing intermediate files (the normal alternative under
many operating systems) won't.

Also, while the pipe transmits a byte-stream, conversions are not necessary.
Most of the existing UNIX utilities operate on text, but it is possible to
pass any datatype through a pipe as long as the receiving program is expecting
it.  Try using fwrite() instead of printf() sometime inside a filter program;
you'll be *amazed* at the performance difference!  The drawback is, this won't
work if the data crosses the boundary between systems with different byte or
halfword ordering conventions, whereas text will work just fine.  It's an
issue of portability versus speed.

>A system which operates on the data values themselves without the intermediate
>file representation can be much more efficient.

There is no "intermediate file representation", unless by "file" you mean
"byte stream".  I don't find it generally useful to confuse these terms.

-- 
	Howard A. Landman
	{oliveb,hplabs}!intelca!mipos3!cpocd2!howard	<- works
	howard%cpocd2%sc.intel.com@RELAY.CS.NET		<- recently flaky
	"Unpick a ninny - recall Mecham"