Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!usc!elroy.jpl.nasa.gov!ames!sun-barr!newstop!eastapps!deitrick!ckollars From: ckollars@deitrick.East.Sun.COM (Chuck Kollars - Sun Technical Marketing - Boston) Newsgroups: comp.protocols.nfs Subject: Re: how many nfsd's should I run? Message-ID: <4541@eastapps.East.Sun.COM> Date: 27 Feb 91 14:55:39 GMT References: <28975@cs.yale.edu> Sender: news@East.Sun.COM Reply-To: ckollars@east.sun.com (Chuck Kollars - Sun Technical Marketing - Boston) Organization: Sun Microsystems, Billerica MA Lines: 101 Both Mike Loukides' book "System Performance Tuning" and Hal Stern's presentation at SUG have raised interest in the question of how many nfsd's/biod's. Here's my summary of public and private answers: In article <28975@cs.yale.edu> anselmo-ed@CS.YALE.EDU (Ed Anselmo) writes: >Is there a magic formula for determining how man nfsd's to run? There's a magic procedure for adjusting the number of nfsd's, and several magic formulas for picking a starting value. Q: So what's the magic procedure? Run 'netstat -s' a few times, and look at the number of "socket overflows" in the "udp:" category. If it's growing, run more nfsd's. If it's zero, you're okay. (If it's large but not changing, probably somebody ran 'spray' on your network yesterday.) Q: So what are the magic formulas for a starting value? Variation 1-- #(disk controllers) + 1 Variation 2-- #(disk controllers) + #(ethernet interfaces) Variation 3-- #(disk controllers) * min(4, #(spindles/controller)) Variation 4-- 4 for a small server, 8 for a midsize server, 16 for a large server Q: Does the number of nfsd's have to be a multiple of 2? No, there can be any number of nfsd's. >"System Performance Tuning" by Loukides says "...4 is appropriate for >most situations. The effects of changing this number are hard to >predict and are often counterintuitive. For various reasons that are >beyone the scope of this book, increasing the number of daemons can >often degrade network performance." > >The SunOS 4.1.1 man page suggests 8 nfsd's. > >What happens when you run too many nfsd's? Server performance may suffer. The amount of degradation depends on your hardware, your software, and just how many "extra" nfsd's you have. In most cases the penalty for erring on the high side is small. On machines with hardware support of multiple contexts (ex: Sun4/SPARC), running more nfsd's than there are hardware contexts will force context switching into software. On machines without software support for signaling semaphores, an NFS request will wake up _all_ the sleeping nfsd's, which will stumble all over each other resulting in a lot of unnecessary context switches and degraded server performance. >Too few? The RPC queue of NFS requests will overflow, resulting in timeouts, retransmissions, backoffs, and hence noticeably degraded client performance. This serious effect can make a large expensive server act like a boat anchor. Q: How important is it to get the number of nfsd's exactly right? Not very. To quote Hal Stern, "This fine-tuning may yield minor improvements, but major performance problems can usually be traced to other bottlenecks or problems." Q: Why do some people run very large numbers of nfsd's, 48 or even 64? 'nhfsstone' benchmark results can often be improved by running a very large number of nfsd's on the server under test. Doing this proves that it's possible. Doing this does not prove that it's helpful on a production server with a mixed or bursty workload. Q: What if the NFS requests are very bursty? Running enough nfsd's to handle the peaks seems to imply running way too many nfsd's most of the time. NFS requests are seldom so bursty that this is a problem. If it is, you could increase "udp_recvspace" in sys/netinet/in_proto.c and rebuild your kernel. Q: Does the number of nfsd's being run affect other things? Yes, be sure MAXUSERS in your kernel config allocates enough kernel resources to run all your nfsd's. Despite what its name seems to imply, MAXUSERS is a "large knob" that controls the size of statically allocated tables in the kernel. On large servers with lots of RAM, turn the knob "way up", otherwise those nfsd's may overflow some kernel table. Q: Is there a similar magic formula or procedure for biod's? Yes, always run the default 4 biod's per client. Changing the number of biod's is usually counterproductive. Reducing the number could cripple client performance. Increasing the number could cripple network or server performance. Changing the number of biod's on the clients either way will require adjusting the number of nfsd's on the server. --- chuck kollars Sun Technical Marketing, located in Sun's Boston Development Center