Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!utgpu!water!watmath!clyde!rutgers!sri-spam!ames!ucbcad!ucbvax!LANL.GOV!cpw%sneezy
From: cpw%sneezy@LANL.GOV.UUCP
Newsgroups: comp.protocols.tcp-ip
Subject: Reassembly queues in TCP
Message-ID: <8705272206.AA01185@sneezy.lanl.gov>
Date: Wed, 27-May-87 18:06:58 EDT
Article-I.D.: sneezy.8705272206.AA01185
Posted: Wed May 27 18:06:58 1987
Date-Received: Sat, 30-May-87 01:23:47 EDT
Sender: daemon@ucbvax.BERKELEY.EDU
Distribution: world
Organization: The ARPA Internet
Lines: 48

Our MILNET Gateway is a VAX-11/750 running Berkeley 4.3 UNIX. Since installing 
4.3 we have experienced a number of crashes of the type:

	panic: out of mbufs: map full

After much searching and accounting for all the message buffers (mbufs), I
determined that this 'panic', in 2 cases, was precipitated by an uncontrolled
stream of 1 byte TCP packets emanating from (in panic #1) a MILNET host 2500
(plus or minus a thousand) miles away.  The actual configurations were:

	panic #1: (VMS login to NRL )             ( a telnet from NRL )
			 -------       -----------     ------     ----
	          tty---|UB RF?	|     |VAX-11/780 |---|MILNET|   |LANL|
			|connect|---tt|VMS Woll...|   |      |---|    |
			 -------       -----------     ------     ----

	panic #2: (VMS login to AFWL )            ( a telnet from AFWL)
			 -------       -----------     ------     ----
	          ibm---|UB ??	|     |VAX-11/780 |---|MILNET|   |LANL|
		  clone	|connect|---tt|VMS Woll...|   |      |---|    |
			 -------       -----------     ------     ----

Interestingly, the host at NRL crashed at the same time our host did.
I did not check out the AFWL case completely.  I would like to know
why these 1 byte packets are coming our way.  In both panics, each byte was
a hexadecimal '0d'.  Could that stand for 0VERdOSE???

The crash results when, a tcp sender begins to pump beaucoup packets out
on the net in assending (sic) sequence each 1 character in length, the network
conveniently loses some small number of them early on, and the receivers window
is greater than the total number of mbufs available.  The tcp input routine
appends these 1 character mbufs on a reassembly queue hoping for the
retransmission of lost packets so that it can make the list available to a
receiving application.  Needless to say, the offending system never sends
the missing pieces, the kernel links whatever remaining system message
buffers there are on the reassembly queue, and finally, some other process
asks for an mbuf with the M_WAIT option, and 4.3 panics in m_clalloc.

I call this the 'silly assembly syndrome'.

I have posted a fix to unix-wizards for the 4.3BSD UNIX folks, which prevents
the receiver from crashing.  However, it is most likely a problem in other TCP
implementations.  It appears to be something the AFWL and NRL, not to mention
other installations, might like fixed.

Any comments?

-Phil Wood (cpw@lanl.gov)