Path: utzoo!mnetor!uunet!husc6!uwvax!umn-d-ub!umn-cs!meccts!mn-at1!alan
From: alan@mn-at1.UUCP (Alan Klietz)
Newsgroups: comp.protocols.tcp-ip
Subject: Re: maximum Ethernet throughput
Message-ID: <412@mn-at1.UUCP>
Date: 9 Mar 88 08:00:02 GMT
References: <8803012359.AA00957@acetes.dec.com>
Reply-To: alan@mn-at1.UUCP (0000-Alan Klietz)
Organization: Roseville, MN
Lines: 72
Summary: Its a general problem

In article <8803012359.AA00957@acetes.dec.com> mogul@DECWRL.DEC.COM (Jeffrey Mogul) writes:
<The best numbers that I am aware of, for communications between a pair
<of hosts, comes from Dave Boggs.  Using a pair of 15 MIPs processors
<with software of his own design he can get about 7.5 Mbits/sec obeying
<the Ethernet spec, or at least 8.6 Mbits/sec if he "cheats" by sending
<4Kbyte packets. 
<
<The limiting factor in his measurements seems to be that his interface
<can only send one packet at a time; i.e., he must process one interrupt
<per transmitted packet, which takes a few hundred microseconds.  The
<interface can receive up to 16 packets using only one interrupt.  With
<a more elaborate interface design, the theoretical limit should be
<attainable.

In general the maximum channel utilization possible by a single host,

assuming no errors, no contention, an unlimited output queue, and

no copy is,

                       t
                        f
              D = ------------
		   t   +   t
		    f       p

where t  is the time required to wiggle over the wire the bits of an
       f
average length data frame over the wire, and t  is the processing
                                              p
overhead time to interrupt the host CPU, switch context, reset the

interface hardware, allocate buffers, and do other (fixed) overhead.

To push the utilization curve to near 1, you have to either make

your data frames big (increase t , or equivalently process more frames
                                f
per interrupt) or make your processing overhead small (decrease t  ).
                                                                 p

Dave Boggs did both things and got good results.  His processing overhead
is 50us, or a 20,000 packets per second.  Impressive.  (The results for
the 4K case show a 66us overhead, from which we can infer that one of
our original assumptions, such as no data copy, were probably wrong,
but the results are still valid.)

But herein lies the problem.  As machines get faster and bigger, with
more pipelining and vectorization, and as the host network software
becomes bigger and more complicated, the per-message processing overhead
gets more expensive.  And yet the data-links are becoming faster and
faster.   FDDI is 100+ mbit/s.  The GaAs version will be 1000+ mbit/s.
The ANSI X3T9 Working Group is developing a spec for a HSC channel rated
at 1600 mbit/s per second.   The importance of absolutely minimizing
the host overhead is something that I think is critical to get any sort
of decent usage of these links (e.g. buffering multiple messages per host
interrupt). 

The problem is that many vendors still think the "critical path" for
getting better performance is the wire.  (Make the wire go faster and
you get better results), when in actuality the critical path is reverting
back the processor.  Chop off the zeros and its not unlike the olden
days of writing a 9600 baud serial driver for a Intel 8080 processor :-)

--
Alan Klietz
Minnesota Supercomputer Center (*)
1200 Washington Avenue South
Minneapolis, MN  55415    UUCP:  alan@mn-at1.k.mn.org
Ph: +1 612 626 1836       ARPA:  alan@uc.msc.umn.edu  (was umn-rei-uc.arpa)

(*) An affiliate of the University of Minnesota