Path: utzoo!news-server.csri.toronto.edu!csri.toronto.edu!mart Newsgroups: comp.dcom.lans From: mart@csri.toronto.edu (Mart Molle) Subject: Re: Ethernet performance degradation? Message-ID: <1991Mar11.133033.25283@jarvis.csri.toronto.edu> Keywords: PC, NFS, Novell, Netware, DOS References: <1991Mar7.220351.9761@uhura1.uucp> <1991Mar7.221028.9883@uhura1.uucp> <45323@ut-emx.uucp> Distribution: usa Date: 11 Mar 91 18:30:33 GMT Lines: 144 spurgeon@atget.cc.utexas.edu (Charles Spurgeon) writes: >In article <1991Mar7.221028.9883@uhura1.uucp>, bryan@uhura1.uucp (Bryan >Curnutt) writes: >|>I've also heard (though I don't know how true it is) that Ethernet >|>network performance degrades drastically if too many nodes are added >|>to the network, and that the number of nodes to do this is relatively >|>small. Is there any truth to this? >Every Ethernet, indeed every LAN no matter what the technology used, has >a different equipment mix and load profile, so your mileage will vary. On >the other hand, there are no special limits to host population/traffic on an >Ethernet other than those listed in the spec (for example, 100 transceivers >per thick Ethernet segment, 30 connections per thin Ethernet segment, >1024 addressable stations per Ethernet system linked with repeaters). It is not as simple as you suggest. There is indeed very strong evidence to show that network performance can degrade drastically when the number of active hosts reaches a threshold that is significantly below 1024. >There have been a number of papers written about performance on various >simulations of "Ethernet like" networks. These simulations seem to be >the ones cited when people are describing "hard limits" in the Ethernet >protocol. For some empirical tests of Ethernet performance that show >what the real Ethernet protocol can deliver, check out the DEC Technical >Report listed below: > Measured Capacity of an Ethernet: Myths and Reality > David R. Boggs, Jeffrey C. Mogul, Christopher A. Kent. > Proceedings of the SIGCOMM '88 Symposium on Com- > munications Architectures and Protocols, ACM > SIGCOMM, Stanford, CA., August 1988, 31 pps. > From the Abstract: > "Ethernet, a 10 Mbit/sec CSMA/CD network, is one of the > most successful LAN technologies. Considerable confu- > sion exists as to the actual capacity of an Ethernet, > especially since some of the theoretical studies have > examined operating regimes that are not characteristic > of actual networks. Based on measurements of an actual > implementation, we show that for a wide class of appli- > cations, Ethernet is capable of carrying its nominal > bandwidth of useful traffic, and allocates the > bandwidth fairly." Why are you citing this paper? Their study was limited to 24 hosts, and hence its findings are irrelevant for the question at hand. Let me give you three reasons to support my claim: 1. Careful examination of the truncated binary exponential backoff algorithm. Consider a worst-case scenario, where a large number (~=1024) of active hosts is beating on the network continuously. According to the back-of- the-envelope ``1/Q'' model introduced in the original 1976 CACM paper by Metcalfe and Boggs, all hosts will quickly spread out their retries until they hit the maximum backoff delay (i.e., uniform retries over the next 1024 ``slot times'') and so the system settles down to stable operation with (roughly) one retry per slot time, giving an asymptotic utilization of (packet length)/ [(packet length) + (e-1)* (slot time)] independent of the number of hosts, as long as its no more than 1024 due to the truncation of the backoff interval. The flaw in this ``1/Q'' model is that the backoff counter gets reset after 15 unsuccessful retries, and thereafter the next several attempts occur very rapidly (as the host ``relearns'' about the congestion problem). It can be shown that in the limit of very high collision rates, the average time between successive retries BY A SINGLE NODE is only about 225 slot times -- so a 1024 host network would have an average of =four= hosts attempting to transmit at every ``slot''. Plugging this value of the traffic generated by each station into an analytical model (e.g., Sohraby, et al., "Comments on `Throughput Analysis for Persistent CSMA Systems'", IEEE Trans. Comm., Feb. 1987) indicates that, depending on the packet size, adding too many hosts causes the percentage of dropped frames to skyrocket and the Ethernet utilization to plummet. For 1024 byte frames, the loss probability shoots up between 10 and 100 hosts, and the utilization drops off between 800 and 1200 hosts; for 64 byte frames the loss probability shoots up between 100 and 600 hosts and the utilization drops off between 100 and 500 hosts. 2. Using simulation (gasp!) to extrapolate the results of the `Myths and Reality' paper to large numbers of hosts. We wrote yet another Ethernet simulator, including all the grubby details we could think of (like the 9.6 usec interpacket gap, 12 byte preamble/ checksum) and configured it to mimic the network configuration reported by Boggs, et al., in the paper above. After validating our simulator by reproducing their experimental data for 1-24 hosts, we increased the number of hosts into the hundreds and observed the same effects at roughly the same numbers of hosts as described above. 3. Cite the experience of someone who has (apparently) actually tried to run an Ethernet with that many hosts. The following article appeared about a year ago in comp.dcom.lans, in response to a similar query about Ethernets with large numbers of hosts. Evidently, the experience at Livermore indicates that 1024 hosts on a single Ethernet `collision domain' is too many... +---------------------------------------------------------------------- | >From: oberman@rogue.llnl.gov (Oberman, Kevin) | Newsgroups: comp.dcom.lans | Subject: Re: 1024 stations on 1 ethernet? | Keywords: ethernet,lan | Message-ID: <51316@lll-winken.LLNL.GOV> | Date: 7 Mar 90 11:15:42 GMT | Sender: usenet@lll-winken.LLNL.GOV | Reply-To: oberman@rogue.llnl.gov | Organization: Lawrence Livermore National Laboratory-Engineering | Lines: 25 | | In article <1990Mar7.012322.18918@granite.cr.bull.com>, piacenti@granite.cr.bull.com (Paul Piacentini) writes... | >my 1st post ... I hope it comes out right | > | > I have read somewhere more than once that up to 1024 stations are allowed | >on 1 ethernet lan. Does this mean 1024 ACTIVE stations, or 1024 adapter | >conections? Does anyone know why this is a limitation? We are rapidly | >approaching this figure, but we have several ethernets connected via | >level 2 bridges. Does this mean we can have up to 1024 stations on each | >side of the bridges? Any info would be appreciated, thanks. | | This limit is technically right, but practically wrong. | | The cause of this limit is the collision backoff alogorithm. When a collision | is detected by a device, the device backs off for a random time. The backoff is | on of 1024 discrete time intervals. If you have more than about 800 active | devices on the Ethernet, thing tend to go foobar. A bridge starts the count | over again. A repeater does not. | | R. Kevin Oberman | Lawrence Livermore National Laboratory | Internet: oberman@icdc.llnl.gov | (415) 422-6955 | | Disclaimer: Don't take this too seriously. I just like to improve my typing | and probably don't really know anything useful about anything. +---------------------------------------------------------------------- Mart L. Molle Computer Systems Research Institute University of Toronto Toronto Canada M5S 1A4 (416)978-4928