Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!usc!apple!agate!violet.berkeley.edu!pete From: pete@violet.berkeley.edu (Pete Goodeve) Newsgroups: comp.sys.amiga.tech Subject: Re: PIPEs Summary: splitting and merging Message-ID: <1990Nov17.072551.5612@agate.berkeley.edu> Date: 17 Nov 90 07:25:51 GMT References: <1990Nov8.101107.5884@zorch.SF-Bay.ORG> <1990Nov10.082242.22949@agate.berkeley.edu> <8283@gollum.twg.com> Sender: usenet@agate.berkeley.edu (USENET Administrator) Organization: University of California, Berkeley Lines: 114 [As others have noted, this thread is now worn thin, but one more time for those who don't yet have it in their kill file...] In <8283@gollum.twg.com> (15 Nov), David S. Herron (david@twg.com) writes: > In article <1990Nov10.082242.22949@agate.berkeley.edu> pete@violet.berkeley.edu (Pete Goodeve) writes: |> [..discussing Kent's suggested multi-way pipe syntax] |> [....] "Fan out pipes" are something that even unix can't do, as |> far as I can see. [.....] > > Unix has the capability of doing this - all the OS facilities are > there it's just that it's hard to represent it in a linear line of > text. Since Unix commands (under /bin/{,c,k}sh) are a linear line > of text this is a problem. [.....] Linear line of text? Tha's funny -- I seem to remember writing a shell script once... (:-)) [Er.. what's a NON-linear line, Teach..? (:-))] Certainly there's nothing in the unix kernel to STOP you doing it, but no shell HAS done it, have they? We were discussing the syntax that a system would need to do such a job, I thought. As to the particular suggested syntax, I think it IS probably rather more complex than it should be. I considered both Kent's and my suggestions as first-cut ones that would need to be hammered on. > > Another problem is -- where might this be used? Any place where you want more than one process to have IMMEDIATE access to data as it is generated. When you're dealing with real-time, this is quite often the case. One of my interests is lab data acquisition and processing, where you might want to both record raw data and process it for display. Then there's things like MIDI, where a single event stream might be patched to several muncher modules. Of more immediate interest is the desire to both display text in a console window and record it in a log file, as well as maybe passing it as data to some other process. > [....] I've lived on Unix for years with just linear pipes. > The few times where it might've been nice to have tree-structures > of piped-together processes has always been in a shell script & it > was pretty easy to use temp files of my own & delete them when done. > [......] > > cmd1 | ( > tee /tmp/some-file; some-other-command ) | cmd2 Yes of course temp files can often do the job, but pipes were invented to cover situations where they AREN'T suitable. Like, as I said, where you need the information NOW -- not when the file is closed. And what about never-ending processes generating information to pass on (or, in the fan-out situation, broadcast)?. The other feature of (unix-shell) pipes of course that has been of major concern in this thread is that -- unlike files, temp or otherwise -- they are unnamed; you can run several copies of a script at once, without them interfering with each other. There are more minor advantages to pipes as well, such as the convenience of NOT having to delete them when done. On the other hand, yes, I would only expect complicated pipe-work to appear in scripts -- not in command lines. My own use of pipes in unix (keyboard) commands seldom extends beyond stuff like "ls -l |grep Nov". Maybe once or twice I've had a two stage filter for something, but never more. On the Amiga most of my commands are scripts [Looking back at my currently visible history list, only a single command isn't a script!] and many of these use (named) pipes. Conversely, I have had occasion to redirect keyboard command I/O through pipes but not often. The Amiga environment is not unix [no comments please...]. Of course my concern is not really at all with "The Way Unix Does It", but what new and BETTER things can we do on the Amiga? And even if it's only OCCASIONALLY very useful, that's probably enough... Once a facility is available, you find applications you never would have thought of otherwise. > [Pardon my re-ordering...!!] > For an ad-hoc creation of tree-structured piles of processes it > seemed you'd want some sort of graphical shell in which you'd have > process names and pipe symbols that you could drag around & connect > up & play around with. Ahh, yes indeed. I have my dreams... > > [.....] If I'm reading > this right you want the input of cmdd to be some combination of > the outputs of cmda, cmdb and cmdc. If each of those are executing > concurrently how do you avoid mixing the outputs? [.....] You don't, of course. There wouldn't be much sense in using such a scheme if you had to keep them separate. But there are lots of cases where the problem doesn't arise (or that's exactly what you want!). For a start, the IP: device passes data by packet, so you don't, for instance, get the characters within lines from different sources being munged together. Then often you can be confident of the order in which data is going to be generated, so merging it is safe. As a particular example, suppose you have a program that takes commands and generates outputs, and you want to merge both into a single stream of command/output sequences. Graphically, you have: Commands --->-------------------> Merged Output | | split merge | | >---> Program ---->/ Because the splitter directs its outputs in a specified order, and because the program always produces its output after an interval -- however small -- you know your output sequence will be correct. -- Pete --