Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!usc!cs.utexas.edu!helios!archone!byron From: byron@archone.tamu.edu (Byron Rakitzis) Newsgroups: comp.unix.shell Subject: Re: Bash, tar, and broken pipe Message-ID: <16290@helios.TAMU.EDU> Date: 17 May 91 14:39:26 GMT References: <586@kepler1.kepler.com> <1991May15.155040.19078@ssd.kodak.com> Sender: usenet@helios.TAMU.EDU Organization: College of Architecture, Texas A&M University. Lines: 79 In article heinz@cc.univie.ac.at writes: >In <1991May15.155040.19078@ssd.kodak.com> weimer@garden.ssd.kodak.com (Gary Weimer (253-7796)) writes: >>In article <586@kepler1.kepler.com>, jwu@kepler.com (Jasper Wu) writes: >>|> I have some problem when using pipelined tar in bash and hope someone >>|> can help me find out why. >>|> >>|> When I do >>|> zcat foo.tar.Z | tar tvfB - >>|> or uncompress < foo.tar.Z | tar tvfB - >>|> >>|> it gives me the table of contents correctly but reports an error >>|> message "Broken pipe" to stderr when it finishes. However, if i add >>|> a "cat" to the pipeline as >>|> cat foo.tar.Z | uncompress | tar tvfB - >>|> then it works fine (i.e., no error message). All commands above work fine >>|> in csh or sh. > >> uncompress -c foo.tar.Z | tar tvf - >> ^^ > >This does *not* fix the problem. [...] >I'm sorry I can't give a solution to the problem - all that I >know is that bash reports a broken pipe if one of the processes >making up the pipe is killed or terminated abnormally (well, it >doesn't have to be terminated abnormally, it only needs to be >just terminated). [...] >It also might be a 'timing prob- >lem' as zcat surely terminates before tar, and this may cause >bash to report the pipe as being broken. Come again? zcat surely terminates before tar? Surely not! The answer to all this is quite simple. When I do: cat /usr/dict/words | sed 10q what happens? sed reads 10 lines of input, and then quits. However, cat does not know it is writing to a pipe, so it keeps dumping stuff to its stdout. Something has to stop it, so Unix has a signal called SIGPIPE. cat therefore dies with the signal "SIGPIPE". Some shells report this with a "broken pipe" message, because the tail of the pipe died before the head. Now, I have not looked at bash source either, but my guess is that the code does not check the status of each exiting pipeline member, because in the line cat foo.tar.Z | uncompress | tar ft - the "uncompress|tar" section of the pipeline will break when tar finishes printing the table of contents of the tar file. Finally, if I type (echo hi; echo there) | sed 1q then a shell will most likely not report any error, since "hi\n" and "there\n" will fit into the pipe buffer. Hope this clears things up a little. (In my opinion, a shell should never report broken pipes, since they are usually a part of normal operation. However, if can be handy to check the exit status of all pipe members. I have written a shell which does this: cat /usr/dict/words | tail -r | exit 42 echo $status prints the output 0 sigpipe 42 but does not report the broken pipe explicitly) -- Byron Rakitzis byron@archone.tamu.edu