Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!cs.utexas.edu!tut.cis.ohio-state.edu!ucbvax!SAYSHELL.UMD.EDU!louie
From: louie@SAYSHELL.UMD.EDU ("Louis A. Mamakos")
Newsgroups: comp.protocols.tcp-ip
Subject: Re: Protocol Design issue
Message-ID: <8912132352.AA18190@sayshell.umd.edu>
Date: 14 Dec 89 02:52:03 GMT
References: <10439@ucsd.Edu>
Sender: daemon@ucbvax.BERKELEY.EDU
Organization: The Internet
Lines: 35

Brian,

MDQS uses a similar mechanism to transfer files from one system to another,
and it works just fine in most cases.  The large problem that we ran into
when implementing MDQS clients and servers on large, ugly unfriendly machines
like Unisys 1100's and IBM 3080's is that there was no easy way to determine
how many bytes were to be sent without parsing the file once.  

MDQS made the assumption that all of the bytes in the file spooled would be
sent as-is over the network.  This is a broken assumption because:

	* It assumes that you can figure out exactly how long in bytes a
	file is.  Some brain damaged file systems make this difficult or
	impossible to do cheaply.

	* It assumes that the bytes are sent as-is over the network, and
	not converted to NVT ASCII, for instance.  This assumption doesn't
	hold on either the IBM or UNISYS mainframes which have, ah,
	interestingly complicated ways of storing "text" in a "file".

We've redesigned the network protocol to be able to send blocks of data,
each prefixed with a byte count.  This allows the sender to have a finite
sized buffer to accumulate the results of the host system representation
(EBCDIC on the IBM, 6 bit FIELDATA or 9 bit ASCII on the UNISYS) to NVT
ASCII translation.  It will no longer be necessary to pre-parse the entire
file just to find out how many bytes will later be shoved across the
network connection.  The additional complexity is minimal.

If you'd like to get clever, the two ends of the connection can negotiate
the largest size buffer that can be used, if required.

I believe that there is a transfer mode defined in FTP which does just
this sort of thing.  Transfer mode BLOCK, I believe.

louie