Path: utzoo!attcan!uunet!mcvax!ukc!reading!minster!bin
From: bin@minster.york.ac.uk
Newsgroups: comp.bugs.sys5
Subject: Serious bug in stdio on some ``System V'' derivatives
Message-ID: <591535074.12423@minster.york.ac.uk>
Date: 29 Sep 88 11:17:54 GMT
Organization: Department of Computer Science, University of York, England
Lines: 46

Some System V systems, particularly those that include Berkeley networking,
are afflicted with a fairly nasty bug if the following happens.

<stdio.h.>:
	#define	_NFILE	20
<sys/param.h>:
	#define	NOFILE	30	/* anything > _NFILE */
and <stdio.h> again:
	#define _bufend(p)      _bufendtab[(p)->_file]

The trouble starts if a process opens some files with stdio, and others
directly, using open, pipe, or whatever.  Eventually, a stdio file
can be opened that has a _file >= _NFILE.  There are two problems:

	1. _file == _NFILE
		System V stdio uses this as a flag for IO to and from
		a string buffer; this convention is used by sprintf/sscanf.
		The internal buffering functions, _filbuf and _flsbuf,
		will then mess up buffer filling & flushing.

		Actually, there is another bug here:  _filbuf
		incorrectly makes a system call for string IO,
		which can slow sscanf down a little...

	2. _file > _NFILE
		_bufendtab is declared _bufendtab[_NFILE+1], to allow
		for the convention mentioned above.  The array entries
		are supposed to give the pointer to the end of the buffer
		for each stdio file.  Unfortunately, in order to allow
		older binary files to be used without recompilation,
		this pointer was not made part of the FILE structure.
		Provided NOFILE <= _NFILE, this is merely inelegant.
		Otherwise, it is a disaster, since the pointer value
		for _file > _NFILE will be whatever junk appears after
		the array in memory, often the stdio buffers.

Berkeley networking has nothing to do with this, of course, except
that ``System V'' systems that have sockets (and perhaps NFS) seem
to be more likely to increase NOFILE.

It's interesting that programs affected by this will often work quite
differently under the control of a debugger.  Careless debuggers leave
some extra file descriptors open to the executable image, core file, etc.

Berkeley systems, including Suns, are unaffected, since they sensibly
put the extra value in the FILE structure.