Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!clyde.concordia.ca!uunet!cs.utexas.edu!wuarchive!brutus.cs.uiuc.edu!apple!agate!saturn!ray
From: ray@saturn.ucsc.edu (Ray Swartz)
Newsgroups: comp.unix.questions
Subject: Why does pipeline work this way?
Keywords: pipeline, grep | cat
Message-ID: <10064@saturn.ucsc.edu>
Date: 15 Dec 89 14:49:44 GMT
Reply-To: ray@saturn.ucsc.edu (Ray Swartz)
Organization: University of California, Santa Cruz
Lines: 28

I have discovered an anomoly that I can't explain and want to know
what is going on.

If you have a large data file (mine is > 3 Megs) and you want to 
use grep to find a line in the file, you have to wait for the grep to 
read through the entire file.  To get around this problem, I decided to
write a shell script that would get piped the output from the grep
and upon reading the first line from the pipe (there will only be one
line of output) terminate the grep process.  Here's the problem, the
shell script (using 'while read data' to read the pipe) doesn't get
the output from grep right away.  In fact, grep still reads
through the entire file before the shell script gets the output.

While studying this further, I tried to see if cat had the same problem.
The command

	grep "pattern" bigfile | cat

also doesn't show output until the grep command completes.

I even tried doing the forking and execing myself and it still waited
until the grep finished.

Yet, when I run grep by itself, the output comes out to the tty as soon as grep
finds a line.  Why is it that when grep prints to a terminal the output
appears unbuffered and when it is going into a pipe, it appears buffered?

Any and all comments appreciated.