Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!att!linac!pacific.mps.ohio-state.edu!zaphod.mps.ohio-state.edu!think.com!linus!agate!usenet
From: pete@violet.berkeley.edu (Pete Goodeve)
Newsgroups: comp.sys.amiga.tech
Subject: Pipe syntax... I think I'd better think it out again...
Message-ID: <1990Nov18.090654.24747@agate.berkeley.edu>
Date: 18 Nov 90 09:06:54 GMT
Sender: usenet@agate.berkeley.edu (USENET Administrator)
Organization: University of California, Berkeley
Lines: 115

Sheesh folks... here we've been pounding on this way long thread about pipe
syntax, separator characters, and such, and nobody -- NOBODY -- has pointed
out that we've all been haring off in exactly the wrong direction!

I got close to it earlier, when I noted that there "aren't many filters
in AmigaDOS", but I didn't follow the implication.  Heck, there're hardly
ANY!  How many of the standard command set will process Standard-Input to
Standard-Output?  Exactly.

Sure, we can write programs that behave as filters, and there are some unix
imports that do -- 'compress' filters like a champ, and I suppose grep does
too (haven't checked) -- but I wonder if that's the best way to do it
anyway.

I mentioned (in another article) a simple unix piped command that I'd used:

            ls -l | grep Nov

Well, OK -- how would you do the equivalent with standard AmigaDOS commands
(LIST and SEARCH), even given that you had a shell with pipe characters?

Using named pipes there SHOULD be no problem [pardon my ego -- I'll use
mine, but any pipe has the same result]:

            run list >IP:xx
            search IP:xx Nov

(Well, it looks good.  Unfortunately if you try it, it doesn't work!
'SEARCH' won't read anything but a file -- not a PIPE: not a CON:!
This is a bug, though -- nothing inherent as far as I can tell. [Use Mat
instead, folks (:-)) -- it's three times as fast as SEARCH anyway...]
To make the idiocy complete, 'SORT' has no qualms about pipes! Replacing
the second command above by:

            sort from IP:xx to *

works fine...)

Leaving aside such little frustrations, there's no way to use un-named
pipes here, unix fashion.  You'd have to redesign the syntax of all the
commands, and I guess it's a little late for that...

We don't have to give up on un-named pipes, though -- just pull our
thinking out of the unix rut.  The pipe/filter concept was a magnificent
stroke of genius when it was thought up, but that was 20 years ago now.
Let's see if we can do better.  While we're at it, I believe we can provide
for the more complex branching pipe-work that Kent Dolan and I were talking
about a few articles back.

Let's begin with an assumption or two.  The main one is that the command
lines as seen by piped programs will be exactly in the form they are at
present.  This means that an input (file) -- maybe an output too --is
usually specified by an argument in the line, identified either by position
or keyword.  We CAN'T expect a program's source of data to be to Standard
Input, although very probably Standard Output will get the results.  This
all means that the program has to open its own inputs (and maybe outputs)
-- it can't always get filehandles from the shell.

I think we have to abandon the popen/filehandle line of thought.
The simple key is to have the shell, PIPE command, or whatever, generate
suitable unique pipe names, and insert them as STANDARD FILENAME STRINGS
into the command line at user specified places.  What we need is a syntax
to specify the insertion points.  Unlike the unix situation, where direction
of dataflow is indicated by sequence on the line, our syntax will have to
say whether each pipe-argument is an input or output.

Some more assumptions:  Most commonly -- but not necessarily -- the source
end of a pipe will be Standard Output; the output end will be an argument
in a command line.  The usual situation will be the unix linear-filter one,
where each process has input from the previous one and outputs to the next,
but we also want to allow for cases where a program may be taking inputs
from more than one stream, and possibly have more than one output as well.
(We already encounter the latter -- have you ever tried to redirect a
program (like C++) that insists on sending to Standard Error?!)

Whatever syntax we choose should be compact and convenient for the
commonest cases (and of course should not get too gross for the other
possibilities...).  I can't claim to have come up with any sort of
satisfactory design at this stage, but I'll throw out some ideas for
thinking about.  For one thing, I'm inclining towards Peter da Silva's
PIPE command as the way to go, at least initially, because it nicely
avoids disastrous consequences from too-quickly-made changes in Shell
syntax: if a particular command breaks, just don't PIPE it!

If we go with the PIPE command, it follows that every piped command
except the last will end with a '+', so that all standard shells will
be happy with it.  I would guess that by default PIPE would assume that
Standard Output from an intermediate command line was to be piped;
if you wanted it to go elsewhere, you'd just redirect it as usual.
(Redirection to the console window would be ">*".)  For an "input
connector" from the previous command we should use a single character
(which BTW would HAVE to be surrounded by spaces -- it's an argument!).
One possibility is our old friend '|'.  As it can't occur alone in a
pattern, or as its first character, there'd be no confusion.  Another
choice might be '+', but that could be nasty for commands that did
arithmetic, or had '-/+' switches [which I kinda like (:-))]
Mmm, has anyone used '@' for anything yet?  (And why not allow the user
choice of the marker character, anyway, with a command line switch, or
maybe better an ENV: variable?)

Using my original example again [and assuming a working SEARCH!] the
suggested syntax would be:

            PIPE list +
            search | Nov

For source ends, and for multi-branch pipes, the syntax would have to
be extended beyond this.  I suggest that various codes beginning with '|'
(or whatever special marker was used) might work.  Thus an output argument
taking the place of Standard Output might be indicated '|+'.  Branches
might be given numeric suffixes: '|1', '|2', '|+1' etc...

Just some meandering thoughts... (and I'm still thinking.)

                                        -- Pete --