Path: utzoo!utgpu!watserv1!watmath!att!rutgers!cs.utexas.edu!uunet!etnibsd!vsh
From: vsh@etnibsd.UUCP (Steve Harris)
Newsgroups: comp.lang.perl
Subject: problem redirecting STDOUT to pipe
Message-ID: <1123@etnibsd.UUCP>
Date: 12 Jul 90 22:26:27 GMT
Organization: Eaton Corp, Semiconductor Equipment Div., Beverly, MA
Lines: 135

Perlers,

	[ I am running perl 3.0, patchlevel 15 (our netfeed has been
	down for a while).  If this problem has been fixed in a more
	recent patch, please let me know. ]


Abstract: Consider the following perl fragments:

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

	print STDERR "before: fileno(STDOUT) = ", fileno(STDOUT), "\n";
	open(STDOUT, ">foo");
	print STDERR "after:  fileno(STDOUT) = ", fileno(STDOUT), "\n";
result:
	before: fileno(STDOUT) = 1
	after:  fileno(STDOUT) = 1

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

	print STDERR "before: fileno(STDOUT) = ", fileno(STDOUT), "\n";
	open(STDOUT, "| dd of=foo");
	print STDERR "after:  fileno(STDOUT) = ", fileno(STDOUT), "\n";
result:
	before: fileno(STDOUT) = 1
	after:  fileno(STDOUT) = 3

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

In the first case, re-opening STDOUT preserves the original fileno, in
the second, it does not.  Why?  Is the way it should behave?


The Whole Story:

I'm writing a (table-driven) script to do backups to an Exabyte tape
drive.  I'm using John Gilmore's public domain tar (which we call
gnutar), which can read the list of files to be archived from stdin.
For the moment, all systems are NFS mounted and treated as part of a
vast virtual filesystem.

I use find to generate the list of files -- depending on conditions,
the archive will be full or incremental.  For incremental backups, I
use "find -newer timestamp_file" to generate the list of files; the
timestamp_file is touch'ed when a full backup is done.

I use bdd, a program from Delta Microsystems (from whom we purchased
our Exabyte), to write to the tape.  Bdd is based on dd, but allows you
to specify both a "bs" parameter (as in dd) and a "odbs" parameter, the
"output device block size".  When bdd processes its final chunk of
data, if that chunk is less than bs bytes, it will write that chunk out
in odbs blocks.  With normal dd, the final chunk is null-padded to bs
bytes.  If bs is 1 Mb, that can be a lot of nulls.

Each partition of each system is handled independently.  E.g., you
could have a full backup of the root partition and an incremental
backup of the /usr partition.  There is a separate timestamp_file in
the root directory of each partition.

Since tape filemarks on the Exabyte are costly (2+ MB, as I recall),
the entire archive for each system is written to a single tape file.
This file is prefixed with a (1024-byte) header describing the contents
of the tar saveset (date, system name; for each partition: type (full
or incremental) and the mod date of the timestamp file.

[ When I get the script working, I'll publish it to the net. ]


Here's an outline of the script (without error checking):

...;	# rewind tape
foreach $sys (@sys_list) {
	...;	# construct partition_list for this sys
	...;	# construct header for this sys
	$pid = open(TAR, "|-");
	if ($pid == 0) {	# child
		open(STDOUT, "| bdd of=$TAPE bs=$bs odbs=$odbs");
		print STDOUT $header;
		exec "gnutar -cvf - -T - -b 1";
	}
	# parent
	foreach $partition (@partition_list) {
		...;	# construct find_args for this sys/partition
		open(FIND, "find $find_args |");
		while (<FIND>) {
			...;		# cannonicize file name
			print TAR $_;	# pass file name to gnutar
		}
	}
	close TAR;	# wait for child to complete
}
...;	# rewind tape


As written above, the script won't work because gnutar has not
inherited the pipe on its stdout.  The "open(STDOUT, ...);" does not
associate file descriptor 1 with the pipe, it just reassigns the name
"STDOUT" to some other file descriptor.  Within perl that's fine, but
gnutar still thinks stdout is file descriptor 1.

I can get around the problem by replacing the child code with:

	if ($pid == 0) {	# child
		open(BDD, "| bdd of=$TAPE bs=$bs odbs=$odbs");
		open(STDOUT, ">&BDD);	# this reassigns STDOUT
		print STDOUT $header;

but then I cannot exec tar, as the tape device does not get closed and
the next attempt to open it (for the backup of the next system) will
fail.  Instead, I have to do (this works):

		system "gnutar -cvf - -T - -b 1";
		close BDD;
		close STDOUT;
	}

If I close BDD before the exec/system, the whole thing blocks.  If I
don't explicitly close both BDD and STDOUT, the next iteration fails
(bdd cannot open $TAPE, presumably it's not closed when the child exits).

I'm not at all sure that causing:
	
	open(STDOUT, "| bdd of=$TAPE bs=$bs odbs=$odbs");

to associate file descriptor 1 with the pipe will solve my problem, but
I wonder, isn't that at least the correct semantics?  And why do I have
to close BDD and STDOUT explicitly?

Comments, Larry?

Just another perl diver, coming up for air.

Steve Harris - Eaton Corp. - Beverly, MA - uunet!etnibsd!vsh
-- 
Steve Harris - Eaton Corp. - Beverly, MA - uunet!etnibsd!vsh