Path: utzoo!utgpu!utstat!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!rutgers!aramis.rutgers.edu!geneva.rutgers.edu!hedrick From: hedrick@geneva.rutgers.edu (Charles Hedrick) Newsgroups: comp.protocols.tcp-ip Subject: Re: NFS Performance through Routers Message-ID: Date: 20 Mar 89 02:47:46 GMT References: <237@alux2.ATT.COM> Organization: Rutgers Univ., New Brunswick, N.J. Lines: 58 Sun NFS does not violate any standard. Now and again when problems like this come up somebody floats this rumor that they somehow transmit "too fast". It just isn't true. It is true that NFS pushes networking technology very hard, but if it doesn't work something on your network is broken. In particular, there is no standard that says something should wait 10 milliseconds between transmissions. If two systems try to transmit at the same time, there is a pattern of random exponential backoff. But if only one host is sending, you may see as little as 9.6 microsecond (I think -- this is from memory) between packets. This stuff is done by the Ethernet controller chip, and Sun uses the same chips as everybody else. I agree with the other response: you've got to find a way for all machines that do broadcasts to use an address that all machines on the network can cope with. In my opinion the safest is 128.92.0.0. Presumably there will be a way to set all machines to use this as their broadcast address. Responding to ARP's with a bogus Ethernet address is a standard technique, and I don't know of anything wrong with it, but it's better to set things up so that you don't need to do that. You should also try setting ipforwarding to 0 in all of your unix kernels: adb -w /vmunix ipforwarding?W 0 ^D This will cause the systems not to attempt to forward stray packets. They may however still send ICMP unreachables back to the source. However this is far better than an ARP, because it isn't a broadcast. If you're willing to do a bit more work in adb on your Sun's, you can make them accept the correct broadcast address. Unfortunately I don't have the exact offset, but if you do adb /vmunix ipintr?i and keep hitting CR, you will eventually find a section of code that calls if_ifwithaddr, compares an address with -1 (or it might show as 0xffffffff), and then calls ip_forward. You want to change the comparison with -1 to compare with the actual broadcast address used on your network. This advice is Sun-specific. Now, as to the rsize and wsize settings. Certainly if a gateway or bridge is dropping packets, you may do better off by using values smaller than the original 8192 default. However 1024 may be going a bit too far. Most Ethernet interfaces can handle two packets in a row, so 2048 would make sense. It does help performance to use as large a number as your hardware can handle reliably. We are able to use 2048 through all of the bridges and routers that we've tried. From what I know of Proteon's hardware, I'd be very surprised if they couldn't handle at least that much. What is critical is how much buffering there is on the Ethernet interface. Newer interfaces often have enough that you can use the full 8192. We have no problem with 8192 with cisco routers that use their new MCI Ethernet interfaces. It's fairly easy to test. Try changing the value and then doing some operation that generates a lot of NFS traffic. Do "nfsstat" before and after the test. Look at "retrans" and "timeout" compared to the total. A couple of percent is OK. My normal test is "cp /server/usr/lib/* /dev/null" Be sure you specify "intr" as one of the mount options, so you can abort the test if something goes wrong. (In fact, I would always specify "intr". I don't know why it isn't the default.)