Path: utzoo!utgpu!news-server.csri.toronto.edu!mailrus!uwm.edu!zaphod.mps.ohio-state.edu!wuarchive!texbell!ficc!peter From: peter@ficc.uu.net (Peter da Silva) Newsgroups: alt.sources.d Subject: Beyond shar (Re: shars and security concerns.) Message-ID: Date: 2 May 90 16:56:25 GMT References: <662@n4hgf.uucp> <1152@chinacat.Unicom.COM> <518@cpsolv.CPS.COM> <1203@chinacat.Unicom.COM> <2641.263ea7ac@mccall.com> Reply-To: peter@ficc.uu.net (Peter da Silva) Organization: Xenix Support, FICC Lines: 786 I also agree that shars are getting out of hand. "shar" was a good idea for its time, but it's gotten too big, too fast. I think it's long past the time for a standard text archive format on Usenet. I vote for the Software Tools format: -h- filename date true_path_name file -h- nextfilename date true_path_name ... Sometimes you see things like this: -h- filename date true_path_name file -t- filename date true_path_name I think there's room to turn this into something we can all live with. Adding the file size, and prefixing all lines with some character, should make things a bit safer. Martin Minow did an implementation of this. It's kind of bare bones, but it should work as a starting place. I'm appending it to this message. In shar format. :-> #! /bin/sh # This is a shell archive. Remove anything before this line, then feed it # into a shell via "sh file" or similar. To overwrite existing files, # type "sh file -c". # The tool that generated this appeared in the comp.sources.unix newsgroup; # send mail to comp-sources-unix@uunet.uu.net if you want that tool. # If this archive is complete, you will see the following message at the end: # "End of shell archive." # Contents: archc.c archx.c readme.txt # Wrapped by peter@ficc.uu.net on Wed May 2 11:49:01 1990 PATH=/bin:/usr/bin:/usr/ucb ; export PATH if test -f 'archc.c' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'archc.c'\" else echo shar: Extracting \"'archc.c'\" \(7785 characters\) sed "s/^X//" >'archc.c' <<'END_OF_FILE' X/* X * A R C H I V E X * X * Create an archive X * X */ X X/*)BUILD $(TKBOPTIONS) = { X TASK = ...ARC X } X*/ X X#ifdef DOCUMENTATION X Xtitle archc text file archive creation Xindex text file archive creation X Xsynopsis X X archc file[s] >archive X Xdescription X X Archc manages archives (libraries) of source files, allowing X a large number of small files to be stored without using X excessive system resources. It copies the set of named X files to standard output in archive format. X X The archx program will recreate the files from an archive. X X Note: there are no checks against the same file appearing X twice in an archive. X Xarchive file format X X Archive files are standard text files. Each archive element is X preceeded by a line of the format: X .s.nf X -h- file.name date true_path_name X .s.f X Note that there is no line or byte count. To prevent problems, X a '-' at the beginning of a record within a user file or embedded X archive will be "quoted" by doubling it. The date and true filename X fields are ignored. On Dec operating systems, file.name is X forced to lowercase. Certain bytes at the beginning of a record are X also prefixed by '-' to prevent mailers from treating them X as commands. X Xdiagnostics X X Diagnostic messages should be self-explanatory X Xauthor X X Martin Minow X X#endif X X#include X#include X X#define unix X#undef vms X X#ifdef vms X#include X#include X#define IO_SUCCESS (SS$_NORMAL | STS$M_INHIB_MSG) X#define IO_ERROR SS$_ABORT X#endif X/* X * Note: IO_SUCCESS and IO_ERROR are defined in the Decus C stdio.h file X */ X#ifndef IO_SUCCESS X#define IO_SUCCESS 0 X#endif X#ifndef IO_ERROR X#define IO_ERROR 1 X#endif X#define EOS 0 X#define FALSE 0 X#define TRUE 1 X Xchar text[513]; /* Working text */ Xchar name[81]; /* Current archive member name */ Xchar pathname[81]; /* Output for argetname() */ Xchar *timetext; /* Time of day text */ Xint verbose = TRUE; /* TRUE for verbosity */ XFILE *infd; /* Input file */ X Xmain(argc, argv) Xint argc; /* Arg count */ Xchar *argv[]; /* Arg vector */ X{ X register int i; /* Random counter */ X register char *fn; /* File name pointer */ X register char *argp; /* Arg pointer */ X int nfiles; X extern char *ctime(); X extern long time(); X long timval; X X time(&timval); X timetext = ctime(&timval); X timetext[24] = EOS; X#ifdef vms X argc = getredirection(argc, argv); X#endif X if (argc <= 1) X fprintf(stderr, "No files to archive?\n"); X#ifdef unix X for (i = 1; i < argc; i++) { X if ((infd = fopen(argv[i], "r")) == NULL) X perror(argv[i]); X else { X strcpy(pathname, argv[i]); X import(); X fclose(infd); X } X } X#else X /* X * Decus C supports fwild/fnext for explicit processing X * of wild-card filenames. X */ X for (i = 1; i < argc; i++) { X if ((infd = fwild(argv[i], "r")) == NULL) X perror(argv[i]); X else { X for (nfiles = 0; fnext(infd) != NULL; nfiles++) { X fgetname(infd, pathname); X import(); X } X fclose(infd); X if (nfiles == 0) X fprintf(stderr, "No files match \"%s\"\n", argv[i]); X } X } X#endif X} X Ximport() X/* X * Add the file open on infd (with file name in pathname) to X * the archive. X */ X{ X unsigned int nrecords; X X fixname(); X nrecords = 0; X printf("-h- %s\t%s\t%s\n", name, timetext, pathname); X while (fgets(text, sizeof text, infd) != NULL) { X switch (text[0]) { X case '-': X case '.': X case '~': X putchar('-'); /* Quote */ X } X fputs(text, stdout); X nrecords++; X } X if (ferror(infd)) { X perror(name); X fprintf(stderr, "Error when importing a file\n"); X } X if (verbose) { X fprintf(stderr, "%u records read from %s\n", X nrecords, pathname); X } X} X Xfixname() X/* X * Get file name (in pathname), stripping off device:[directory] X * and ;version. The archive name ("file.ext") is written to name[]. X * On a dec operating system, name is forced to lowercase. X */ X{ X register char *tp; X register char *ip; X char bracket; X extern char *strrchr(); X X#ifdef unix X /* X * name is after all directory information X */ X if ((tp = strrchr(pathname, '/')) != NULL) X tp++; X else X tp = pathname; X strcpy(name, tp); X#else X strcpy(name, pathname); X if ((tp = strrchr(name, ';')) != NULL) X *tp = EOS; X while ((tp = strchr(name, ':')) != NULL) X strcpy(name, tp + 1); X switch (name[0]) { X case '[': bracket = ']'; X break; X case '<': bracket = '>'; X break; X case '(': bracket = ')'; X break; X default: bracket = EOS; X break; X } X if (bracket != EOS) { X if ((tp = strchr(name, bracket)) == NULL) { X fprintf(stderr, "? Illegal file name \"%s\"\n", X pathname); X } X else { X strcpy(name, tp + 1); X } X } X for (tp = name; *tp != EOS; tp++) { X if (isupper(*tp)) X *tp = tolower(*tp); X } X#endif X} X X#ifdef unix Xchar * Xstrrchr(stng, chr) Xregister char *stng; Xregister char chr; X/* X * Return rightmost instance of chr in stng. X * This has the wrong name on some Unix systems. X */ X{ X register char *result; X X result = NULL; X X do { X if (*stng == chr) X result = stng; X } while (*stng++ != EOS); X return (result); X} X#endif X X/* X * getredirection() is intended to aid in porting C programs X * to VMS (Vax-11 C) which does not support '>' and '<' X * I/O redirection. With suitable modification, it may X * useful for other portability problems as well. X */ X Xstatic int Xgetredirection(argc, argv) Xint argc; Xchar **argv; X/* X * Process vms redirection arg's. Exit if any error is seen. X * If getredirection() processes an argument, it is erased X * from the vector. getredirection() returns a new argc value. X * X * Warning: do not try to simplify the code for vms. The code X * presupposes that getredirection() is called before any data is X * read from stdin or written to stdout. X * X * Normal usage is as follows: X * X * main(argc, argv) X * int argc; X * char *argv[]; X * { X * argc = getredirection(argc, argv); X * } X */ X{ X#ifdef vms X register char *ap; /* Argument pointer */ X int i; /* argv[] index */ X int j; /* Output index */ X int file; /* File_descriptor */ X X for (j = i = 1; i < argc; i++) { /* Do all arguments */ X switch (*(ap = argv[i])) { X case '<': /* ': /* >file or >>file */ X if (*++ap == '>') { /* >>file */ X /* X * If the file exists, and is writable by us, X * call freopen to append to the file (using the X * file's current attributes). Otherwise, create X * a new file with "vanilla" attributes as if X * the argument was given as ">filename". X * access(name, 2) is TRUE if we can write on X * the specified file. X */ X if (access(++ap, 2) == 0) { X if (freopen(ap, "a", stdout) != NULL) X break; /* Exit case statement */ X perror(ap); /* Error, can't append */ X exit(IO_ERROR); /* After access test */ X } /* If file accessable */ X } X /* X * On vms, we want to create the file using "standard" X * record attributes. create(...) creates the file X * using the caller's default protection mask and X * "variable length, implied carriage return" X * attributes. dup2() associates the file with stdout. X */ X if ((file = creat(ap, 0, "rat=cr", "rfm=var")) == -1 X || dup2(file, fileno(stdout)) == -1) { X perror(ap); /* Can't create file */ X exit(IO_ERROR); /* is a fatal error */ X } /* If '>' creation */ X break; /* Exit case test */ X X default: X argv[j++] = ap; /* Not a redirector */ X break; /* Exit case test */ X } X } /* For all arguments */ X return (j); X#else X /* X * Note: argv[] is referenced to fool the Decus C X * syntax analyser, supressing an unneeded warning X * message. X */ X return (argv[0], argc); /* Just return as seen */ X#endif X} X X X END_OF_FILE if test 7785 -ne `wc -c <'archc.c'`; then echo shar: \"'archc.c'\" unpacked with wrong size! fi # end of 'archc.c' fi if test -f 'archx.c' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'archx.c'\" else echo shar: Extracting \"'archx.c'\" \(7837 characters\) sed "s/^X//" >'archx.c' <<'END_OF_FILE' X/* X * A R C H X X * X * Archive extraction X * X */ X X/* X * Note: the )BUILD comment is extracted by a Decus C tool to construct X * system-dependent compiler command lines. X * X * Text inside #ifdef DOCUMENTATION is converted to runoff by a X * Decus C tool. X */ X X/*)BUILD $(TKBOPTIONS) = { X TASK = ...ARX X } X*/ X X#ifdef DOCUMENTATION X Xtitle archx text file archiver extraction Xindex text file archiver extraction X Xsynopsis X X archx archive_files X Xdescription X X Archx manages archives (libraries) of source files, allowing X a large number of small files to be stored without using X excessive system resources. Archx extracts all files from X an archive. X X If no archive_name file is given, the standard input is read. X Archive header records are echoed to the standard output. X Xarchive file format X X Archive files are standard text files. Each archive element is X preceeded by a line of the format: X .s.nf X -h- file.name date true_name X .s.f X Note that there is no line or byte count. To prevent problems, X a '-' at the beginning of a record within a user file or embedded X archive will be "quoted" by doubling it. The date and true filename X fields are ignored. On some operating systems, file.name is X forced to lowercase. The archive builder (archc) may prefix X other characters by '-'. X X If the first non-blank line of an input file does not X begin with "-h", the text will be appended to "archx.tmp" X This is needed if archives are distributed by mail X and arrive with initial routing and subject information. X Xdiagnostics X X Diagnostic messages should be self-explanatory X Xauthor X X Martin Minow X Xbugs X X#endif X X#include X#include X#ifdef vms X#include X#include X#define IO_SUCCESS (SS$_NORMAL | STS$M_INHIB_MSG) X#define IO_ERROR SS$_ABORT X#endif X/* X * Note: IO_SUCCESS and IO_ERROR are defined in the Decus C stdio.h file X */ X#ifndef IO_SUCCESS X#define IO_SUCCESS 0 X#endif X#ifndef IO_ERROR X#define IO_ERROR 1 X#endif X X#define EOS 0 X#define FALSE 0 X#define TRUE 1 X X/* X * The following status codes are returned by gethdr() X */ X#define DONE 0 X#define GOTCHA 1 X#define NOGOOD 2 X Xchar text[513]; /* Working text line */ Xchar name[81]; /* Current archive member name */ Xchar filename[81]; /* Working file name */ Xchar arfilename[81]; /* Archive file name */ Xchar fullname[81]; /* Output for argetname() */ Xint verbose = TRUE; /* TRUE for verbosity */ Xint first_archive; /* For mail header skipping */ X Xmain(argc, argv) Xint argc; /* Arg count */ Xchar *argv[]; /* Arg vector */ X{ X register int i; /* Random counter */ X int status; /* Exit status */ X X#ifdef vms X argc = getredirection(argc, argv); X#endif X status = IO_SUCCESS; X if (argc == 1) X process(); X else { X for (i = 1; i < argc; i++) { X if (freopen(argv[i], "r", stdin) != NULL) X process(); X else { X perror(argv[i]); X status = IO_ERROR; X } X } X } X exit(status); X} X Xprocess() X/* X * Process archive open on stdin X */ X{ X register char *fn; /* File name pointer */ X register FILE *outfd; X register int i; X X text[0] = EOS; X while ((i = gethdr()) != DONE) { X switch (i) { X case GOTCHA: X if ((outfd = fopen(name, "w")) == NULL) { X perror(name); X fprintf(stderr, "Can't create \"%s\"\n", name); X arskip(); X continue; X } X break; X X case NOGOOD: X fprintf(stderr, "Missing -h-, writing to archx.tmp\n"); X fprintf(stderr, "Current text line: %s", text); X strcpy(name, "archx.tmp"); X if ((outfd = fopen(name, "a")) == NULL) { X perror(name); X fprintf(stderr, "Cannot append to %s\n", name); X arskip(); X continue; X } X break; X } X arexport(outfd); X fclose(outfd); X } X} X Xint Xgethdr() X/* X * If text is null, read a record, returning to signal input state: X * DONE Eof read X * NOGOOD -h- wasn't first non-blank line. Line is in text[] X * GOTCHA -h- found, parsed into name. X */ X{ X register char *tp; X register char *np; X Xagain: if (text[0] == EOS X && fgets(text, sizeof text, stdin) == NULL) X return (DONE); X if (text[0] == '\n' && text[1] == EOS) { X text[0] = EOS; X goto again; X } X if (text[0] != '-' X || text[1] != 'h' X || text[2] != '-') X return (NOGOOD); X for (tp = &text[3]; isspace(*tp); tp++) X ; X for (np = name; !isspace(*tp); *np++ = *tp++) X ; X *np = EOS; X return (GOTCHA); X} X Xarskip() X/* X * Skip to next header X */ X{ X while (fgets(text, sizeof text, stdin) != NULL) { X if (text[0] == '-' && text[1] == 'h' && text[2] == '-') X return; X } X text[0] = EOS; /* EOF signal */ X} X Xarexport(outfd) Xregister FILE *outfd; X/* X * Read secret archive format, writing archived data to outfd. X * Clean out extraneous ,'s X */ X{ X register char *tp; X unsigned int nrecords; X X printf("Creating \"%s\", ", name); X nrecords = 0; X while (fgets(text, sizeof text, stdin) != NULL) { X tp = &text[strlen(text)]; X if (tp > &text[1] && *--tp == '\n' && *--tp == '\r') { X *tp++ = '\n'; X *tp = EOS; X } X if (text[0] == '-') { X if (text[1] == 'h') X goto gotcha; X fputs(text+1, outfd); X } X else { X fputs(text, outfd); X } X nrecords++; X } X text[0] = EOS; Xgotcha: printf("%u records\n", nrecords); X if (ferror(stdin) || ferror(outfd)) X printf("Creation of \"%s\" completed with error\n", name); X} X X/* X * getredirection() is intended to aid in porting C programs X * to VMS (Vax-11 C) which does not support '>' and '<' X * I/O redirection. With suitable modification, it may X * useful for other portability problems as well. X */ X X#ifdef vms Xstatic int Xgetredirection(argc, argv) Xint argc; Xchar **argv; X/* X * Process vms redirection arg's. Exit if any error is seen. X * If getredirection() processes an argument, it is erased X * from the vector. getredirection() returns a new argc value. X * X * Warning: do not try to simplify the code for vms. The code X * presupposes that getredirection() is called before any data is X * read from stdin or written to stdout. X * X * Normal usage is as follows: X * X * main(argc, argv) X * int argc; X * char *argv[]; X * { X * argc = getredirection(argc, argv); X * } X */ X{ X register char *ap; /* Argument pointer */ X int i; /* argv[] index */ X int j; /* Output index */ X int file; /* File_descriptor */ X X for (j = i = 1; i < argc; i++) { /* Do all arguments */ X switch (*(ap = argv[i])) { X case '<': /* ': /* >file or >>file */ X if (*++ap == '>') { /* >>file */ X /* X * If the file exists, and is writable by us, X * call freopen to append to the file (using the X * file's current attributes). Otherwise, create X * a new file with "vanilla" attributes as if X * the argument was given as ">filename". X * access(name, 2) is TRUE if we can write on X * the specified file. X */ X if (access(++ap, 2) == 0) { X if (freopen(ap, "a", stdout) != NULL) X break; /* Exit case statement */ X perror(ap); /* Error, can't append */ X exit(IO_ERROR); /* After access test */ X } /* If file accessable */ X } X /* X * On vms, we want to create the file using "standard" X * record attributes. create(...) creates the file X * using the caller's default protection mask and X * "variable length, implied carriage return" X * attributes. dup2() associates the file with stdout. X */ X if ((file = creat(ap, 0, "rat=cr", "rfm=var")) == -1 X || dup2(file, fileno(stdout)) == -1) { X perror(ap); /* Can't create file */ X exit(IO_ERROR); /* is a fatal error */ X } /* If '>' creation */ X break; /* Exit case test */ X X default: X argv[j++] = ap; /* Not a redirector */ X break; /* Exit case test */ X } X } /* For all arguments */ X return (j); X} X#endif X END_OF_FILE if test 7837 -ne `wc -c <'archx.c'`; then echo shar: \"'archx.c'\" unpacked with wrong size! fi # end of 'archx.c' fi if test -f 'readme.txt' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'readme.txt'\" else echo shar: Extracting \"'readme.txt'\" \(1420 characters\) sed "s/^X//" >'readme.txt' <<'END_OF_FILE' XThis is a suggested replacement for shar. It is based on Xthe archive program in Kernighan and Plauger's Software Tools, Xbut has been heavily simplified. X XIt has the following advantages over shar: X X1. it is not tied to Unix -- thus VMS users can unpack files without X excessive effort. Archc and archx should run without change on X all Unix and Unix lookalike systems, as well as on VMS (VaxC) X and all PDP-11 Decus C systems. It has been in use for over 6 X years. X X2. it does not execute the distributed image, but interprets it. This X means that trojan horses cannot be concealed in distributions. X X3 The distribution file can be edited without damaging the archive. X (Also, embedded archives can be handled). X XIt has the following disadvantages: X X1. It is not as flexible as shar -- it cannot create directories or X access any other Unix system services. X X2. There is no checksum capability (it appears impossible to implement X checksumming in a system-independent manner). X XTo use, save this message. Then, use your favorite editor to extract Xarchx.c (delimited by lines beginning with "-h-" in column 1). Then Xcompile archx and run it using the command: X archx XIt should produce readme.txt, archx.c, and archc.c. XManual pages can be produced by extracting the text delimited by X #ifdef DOCUMENTATION X ... X #endif X XPlease report problems to the author: X XMartin Minow Xdecvax!minow X END_OF_FILE if test 1420 -ne `wc -c <'readme.txt'`; then echo shar: \"'readme.txt'\" unpacked with wrong size! fi # end of 'readme.txt' fi echo shar: End of shell archive. exit 0 -- _--_|\ `-_-' Peter da Silva. +1 713 274 5180. / \ 'U` Have you hugged your wolf today? \_.--._/ Disclaimer: commercial solicitation by email to this address v is acceptable.