Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!seismo!ut-sally!husc6!sri-unix!teknowledge-vaxc!dplatt
From: dplatt@teknowledge-vaxc.ARPA (Dave Platt)
Newsgroups: comp.unix.wizards,comp.protocols.tcp-ip
Subject: Re: IP fragmentation, and how to avoid it
Message-ID: <13179@teknowledge-vaxc.ARPA>
Date: Fri, 29-May-87 13:04:44 EDT
Article-I.D.: teknowle.13179
Posted: Fri May 29 13:04:44 1987
Date-Received: Sun, 31-May-87 07:48:58 EDT
Organization: Teknowledge, Inc., Palo Alto CA
Lines: 70
Xref: mnetor comp.unix.wizards:2511 comp.protocols.tcp-ip:298
Posting-Front-End: GNU Emacs 18.41.3 of Tue Apr  7 1987 on teknowledge-vaxc (berkeley-unix)


About three weeks ago I posted a query concerning an IP-fragmentation
problem that I had encountered on my Sun workstation.  I've received a
really astounding amount of assistance from folks on the net, and have
been able to zap the problem.  Several people have asked me to
summarize my findings and the answers I received from informed
netfolks... so, here goes.

- The original symptom of the problem was that SMTP connections would
  hang, and then abort with a network-read timeout, while sending
  large messages to a few hosts on the Internet.  Other hosts
  (including those of the same type as the affected systems) were not
  affected.

- Several people suggested that I check to ensure that my Ethernet
  interface was configured with the -trailers option (it is).

- The problem was triggered by the fact that the MTU of my Sun's
  Ethernet interface (1500 bytes) was less than the MTU of our ARPANET
  gateway's IMP interface (1006 bytes).  This situation caused the
  TCP/IP packets sent by my Sun to be fragmented as they passed
  through the gateway.

- The fragmented packets would occasionally fail to be reassembled
  upon reception.  Some hosts apparently don't implement IP-packet
  reassembly (or don't do it reliably).  Also, I'm told that there is
  a bug in BSD 4.2 UNIX (and possibly in 4.3 as well) that prevents
  BSD systems from successfully fragmenting an already-fragmented IP
  packet.  Thus, if a 1006-byte fragment from our net's gateway had to
  be refragmented to fit within the MTU of the destination host's
  network, the new fragments would be malformed and could not be
  successfully reassembled.

- One method for working around the problem is to reduce the Sun's
  Ethernet MTU to <= 1006 bytes, so that our gateway won't have to
  fragment the packets.  I was able to locate the constant 1500 in the
  "ether_attach()" function in /vmunix, and patch it down to 1000
  bytes with adb; booting with the patched /vmunix resolved the
  problem.  Charles Hedrick posted the source for a small program that
  can change the MTU of the interface "on the fly", and it also works
  like a charm;  it's the method I'm now using.

  Reducing the Ethernet MTU increases the number of packets needed to
  complete NFS RPCs, and thus increases the overhead;  NFS continues
  to work just fine.  I've been warned that decreasing the MTU will
  probably break ND, but as I don't use it I don't really care.

- Another method for fixing the problem is persuading TCP to use a
  smaller segment size, so that the packets that it sends will not
  exceed the 1006-byte limit.  I tried patching the 1024-byte MSS in
  tcp_output() to a smaller size (512 bytes), but this did not appear
  to work.  I'm not sure why, as I have no sources for the SunOS 3.2
  version of BSD 4.2 TCP.

  Many people have pointed out that BSD 4.3 TCP makes a better choice
  of MSS, based on the MTU of the interface and on whether the packets
  will be routed through a gateway (a 512-byte MSS is used if the
  packets are sent to any non-local destination).  The BSD 4.3
  enhancements have been incorporated into SunOS 3.4, which is due to
  be shipped Real Soon Now according to our Sun sales-rep.  I FTP'ed
  the BSD 4.3 source for TCP from seismo (thanks, rick!) and can see
  the additional logic;  I haven't tried to retrofit the new TCP into
  SunOS 3.2 or patch in equivalent code due to lack of time and lack
  of urgency.

So... I've got a good workaround for the problem (reducing the MTU),
and the problem will go away once I install SunOS 3.4 with the BSD 4.3
enhancements to TCP.  Happy ending.

MANY thanks to all of the people on the net who have sent suggestions,
hints, and reports of similar problems elsewhere!