Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!ucbvax!proteon.com!mac From: mac@proteon.com (Michael A. Curtis) Newsgroups: comp.protocols.tcp-ip Subject: NFS Performance through Routers Message-ID: <8903211706.AA26506@monk.proteon.com> Date: 21 Mar 89 17:06:00 GMT References: <237@alux2.ATT.COM> Sender: daemon@ucbvax.BERKELEY.EDU Organization: The Internet Lines: 103 John, Here are a couple of things to try to help with the throughput problem with your SUN's running NFS. Some of this is a rehash of what you have already seen, hopefully with a better explanation. Number 1) deals with configuring the P4200's to quiet down the ARP storms that the SUN's are causing. It is also recommended that you also go into each SUN which is not acting as a gateway and turn off IP forwarding. By turning off the IP forwarding, you will be able to limit the size of the ARP storm. However, you must realize that you will still see a series of ICMP host unreachable messages for each RIP packet. The real fix for this situation is to have all machines configured with the same broadcast address, ie 0's or 1's. Again, the future implications must be considered as you may be able (in fact, required if you are using systems which are running 4.2 BSD) to set the broadcast to 0's on all machines; however, you will eventually have to change all machines to a 1's broadcast as BSD Unix cleans this issue up. Number 2) deals with setting window size for NFS in an effort to improve performance. 1) - Broadcasts By default, the Sun machines will try and forward any packet they receive that: A. Is not for them, but is on the same (sub)network (depending on what rev SW they are running). B. Is on a different network. This behaviour occurs irregardless of the method of packet reception (addressed, multicast, or broadcast). (This is a generic problem with all hosts using networking software derived from the 4.2 BSD distribution.) The broadcast storm comes from the fact that they are not recognizing the IP destination as the broadcast address for that (sub)network. The older Sun's only recongnize 0.0.0.0 or net.0. When they see net.255, they just think that's another host on the (sub)network (no different from net.254), and forward it. This is why we let you set the broadcast so many different ways, and why the default fill is still 0. To change the broadcast fill you need to go into the CONFIG process (T 6) and to process 0 ("IP Config>" prompt). Type "set broadcast". You might try a LOCAL WIRE broadcast, but first just set the fill pattern to 0 with a NETWORK broadcast. The problem with NFS is that it sends large UDP packets (default 8192 bytes), which are then fragmented at the IP level. The problem with this is that a large number of packets (6) are sent in a row. If you send a burst of 6 packets, we will probably miss one or more, especially on a heavily loaded ethernet. Even though the host will retransmit, the retransmitted burst has a different IP unique ID, so the IP fragments from the two transmissions cannot be combined. This results in having to retransmit until all 6 get through at once. 2) - NFS window size The solution is to shrink the size of the UDP writes. Sun also has to do with when running through a Sun as a router, or when using a Sun-2 as a file server for a fast machine like a Sun-3 or Sun-4. Here is an old message on the subject: How to slow down NFS is something we've known you could do, but not known how for some time. It appears to be important when running NFS through our p4200, which is why I'm posting it here. Someone might also want to post it to the p4200 mailing list, if it is still an outstanding issue in the field. >From Sun Software Technical Bulletin, February 1987, in the customer buglist section: ------- 135. Synopsis: fstab entry for sun-3 mounting from a 3com sys not documented Release: 3.2 Description: When a Sun-3 system with a Sun Ethernet board NFS mounts a filesystem from a system (sun-2 only? I don't know) that has a 3com ethernet board, extra entries are required on the /etc/fstab line to make it work successfully. The Sun-3/ie machine pumps out packets so fast that the 3com system can't keep up. The rsize and wsize of packets has to be limited, or else you get lots and lots of retransmissions and "server not responding" msgs. This was documented in the 2.0 to 3.0 Change notes (or release notes), but is not noted in the 3.2 manual set. It needs to be in the System Administration manual in the section covering entworking and NFS. Work Around: The line in /etc/fstab needs to be of this form: 3com_machine:/usr /usr nfs rw,noquota,soft,rsize=2048,wsize=2048 0 0 (the rsize and wsize entries are what is relevant) Additionally, we have no record of you ever calling in to Proteon for support or assistance. You should be aware that you can request help directly from Proteon either via telephone (508-898-3100) or electronic mail (to bug-cgw@proteon.com) as detailed in the support pamphlet. Additionally, there is a P4200 users group which you can address mail to. To become a member, send a request stating so to p4200-request@devvax.tn.cornell.edu. To send mail to this list, send it to p4200@devvax.tn.cornell.edu.