Path: utzoo!yunexus!geac!syntron!jtsv16!uunet!husc6!bloom-beacon!tut.cis.ohio-state.edu!mailrus!ames!pasteur!ucbvax!RICE.EDU!retrac
From: retrac@RICE.EDU (John Carter)
Newsgroups: comp.protocols.tcp-ip
Subject: Re: 4BSD TCP Ethernet Throughput
Message-ID: <573.retrac.titan@Rice>
Date: 27 Oct 88 16:08:04 GMT
Article-I.D.: Rice.573.retrac.titan
Sender: daemon@ucbvax.BERKELEY.EDU
Organization: The Internet
Lines: 66

Van,

    I've made similar measurements on the similar machines, and come to
roughty the same conclusions.  My measurements are in the context of
the V operating systems interkernel protocols, but for raw hardware speed
comparisons, this shouldn't matter.

    Between two SUN-3/50 or SUN-3/60's (both with the LANCE interface),
I can sustain about 8.2 Mbps user-to-user performance (not source/chargen,
sink/discard).  From a SUN-3/180 (somewhat slower than the 2/180, but
with the same Intel ethernet interface) to a SUN-3/60, I can sustain
slightly more than 6.0 Mbps user-to-user performance.  All these
measurements are for 1024 byte data packets, with 80 byte V interkernel
headers (ouch!!!) and 14 byte ethernet headers.  Factoring the headers
in, the SUN-3/50 -> SUN-3/50 throughput is 9.0 Mbps and SUN-3/180 ->
SUN-3/50 throughput is 6.5 Mbps.  Roughly the same raw numbers...

    The SUN-2/50's (which use the Intel interface, but are significantly
slower) can maintain around 4.7-5.1 Mbps in or out.  These are very rough,
since I haven't fully debugged the implementation on the 2's.

    [ The following is opinion and shouldn't be construed as gospel. ]

    I also have only put a little bit of effort into determining the exact
cause of the disparity.  I had made the same decision you had regarding
the 82586's DMA ability, namely, it isn't very good (and can only sustain
60-70% of the network performance).  You conjectured that the interface
takes a long time to recover from interruptions.  I hadn't seen too many
collisions, so I hadn't thought of that, but it seems to fit with
some other observations I've made concerning the interface.  The Intel
interface tends to drop packets reasonably frequently when receiving large
packet bursts (blasts), presumably because of its inability to DMA in to
memory fast enough.  Another problem I have had, which seems to be caused
by the interface, is that it takes a relatively long time for the
interface to interrupt the processor when an event occurs (either a packet
reception completing, or a transmission completing).  An annoying
"feature" of the interface is the fact that you can't have receive buffers
start on odd boundaries (I suppose that they wanted to simplify the DMA
design).

    Finally, despite the great effort put in to designing a "programmable"
interface, I don't really think it was much easier to get to do what we
needed than the LANCE was.  True, it has a few less annoying
programmability problems (e.g., a less obscure method of selectively
accepting multicast packets, and the decision to not append the useless
harldware CRC to the end of each packet, thus requiring its handling
[which is particularly painful for optimistic blast, because I don't want
it to get redirected in to user memory, sigh]), but the overall decline in
raw performance overshadows these issues.  Heck, you only have to program
the thing once, performance lives forever (and is what counts)!

John

[And for anyone out there reading the cc'd copy of this, I'd like to add
 my voice to the call for a better ethernet interface design.  The
 existing ones are quite lacking in many ways, and put far too much
 load on the processor.  If you're going to design one, I have some ideas
 for what would be useful.  There were some interesting designs presented
 at Sigcomm '88 which address some of the problems I have with current
 designs, though not all of them.]

    Finally, since I cc'd this to tcp-ip, all of the above is
    Copyright (c) 1988 by John Carter.  This way, I don't get
    misquoted and, more importantly, Van doesn't get misquoted
    by my references to his work.