Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!seismo!husc6!rutgers!lll-crg!nike!ucbcad!ucbvax!YALE.ARPA!LEICHTER-JERRY
From: LEICHTER-JERRY@YALE.ARPA
Newsgroups: mod.computers.vax
Subject: Re: Stream_LF (again) LONG!
Message-ID: <8611022157.AA00980@ucbvax.Berkeley.EDU>
Date: Sun, 2-Nov-86 16:57:10 EST
Article-I.D.: ucbvax.8611022157.AA00980
Posted: Sun Nov  2 16:57:10 1986
Date-Received: Tue, 4-Nov-86 01:44:25 EST
Sender: daemon@ucbvax.BERKELEY.EDU
Reply-To: <LEICHTER-JERRY@YALE.ARPA>
Organization: The ARPA Internet
Lines: 68
Approved: info-vax@sri-kl.arpa


    Re: Stream Files again.
    [The author reports a variety of problems with the following file:]
    
    Directory SYS$USERDEVICE:[LENT.XROFF.TEST]
    
    K.C;1                         File ID:  (19345,18,0)       
    Size:           27/27         Owner:    [VMS,LENT]
    File organization:  Sequential
    File attributes:    Allocation: 27, Extend: 0, Global buffer count: 0
                        No version limit
    Record format:      Stream_LF
    Record attributes:  Carriage return carriage control
    
    FILE HEADER
    
    	File Spec: SYS$USERDEVICE:[LENT.XROFF.TEST]K.C;1

    RMS FILE ATTRIBUTES
    
    	File Organization: sequential
    	Record Format: stream-LF
    	Record Attributes:   carriage-return
    	Maximum Record Size: 0
    	Longest Record: 0
    	Blocks Allocated: 27, Default Extend Size: 0
    	End-of-File VBN: 27, Offset: %X'008A'
    	Global Buffer Count: 0
    ***  VBN 16:  Last stream record does not contain a delimiter.
    ***  Drastic structure error precludes further analysis.
A STREAM_LF file is, to RMS, a record-oriented file - EVERYTHING is a record-
oriented file!  For a STREAM_LF file, records are terminated by linefeeds;
that is, the first record runs from byte 0 to the first LF; the second from
just after the LF to the next LF; and so on.  The last record in the file
is terminated by the end of the file, if there are no further LF's.

K.C appears to contain no LF's at all.  This produces two problems:

	1.  Since there are no LF's, the file appears to consist of a single
		very long record - about 27*512 bytes long (the final block
		may not be full - there's a "first free byte in last block"
		value in the header - I think that's what the "Offset " is.
		All the usual utilities set up buffers of some "reasonable"
		size - 1024 bytes, or whatever.  When they then try to read
		this file, their record buffers overflow and they die.

		Note that there's a field in the header that indicates how
		long the longest record in the file is.  Programs can and
		often do use this to allocate record buffers.  In this case,
		however, the value is 0, meaning "unspecified", so programs
		have to guess.  (A value of 0 here is usually the result of
		doing block operations to a record file.)

	2.  I said above that the last record ends at a final LF, or at the
		end of the file.  Well, that's not QUITE true.  There's a
		bug in RMS that causes it to die - killing your process -
		if you try to read a STREAM_LF file with no trailing LF.
		(This bug has been reported many times; I don't know if it's
		fixed in V4.4 or V4.5 or V4.?.  It's certainly in V4.3.)
		Your processes aren't running into the bug apparently because
		RMS is running into the "record too long for user's buffer"
		problem before reaching the end of the file.  ANALYZE/RMS
		IS running into the problem, but apparently is able to deal
		with it, at least to the point of given you some kind of
		description of the problem, rather than dying.

							-- Jerry
-------