Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!utgpu!water!watmath!clyde!rutgers!mcnc!ece-csc!ncrcae!ncr-sd!hp-sdd!hplabs!ucbvax!A.ISI.EDU!CERF
From: CERF@A.ISI.EDU.UUCP
Newsgroups: comp.protocols.tcp-ip
Subject: Re: TCP performance limitations
Message-ID: <[A.ISI.EDU]30-Sep-87.09:40:42.CERF>
Date: Wed, 30-Sep-87 09:40:00 EDT
Article-I.D.: <[A.ISI.EDU]30-Sep-87.09:40:42.CERF>
Posted: Wed Sep 30 09:40:00 1987
Date-Received: Fri, 2-Oct-87 02:40:41 EDT
References: <AUERBACH@CSL.SRI.COM>
Sender: daemon@ucbvax.BERKELEY.EDU
Distribution: world
Organization: The ARPA Internet
Lines: 59

Karl,

I have been doing some work in this area to predict potential
performance at 100 Mb/s FDDI rates. First, you can expect a
TCP to execute about 1000-1200 instructions per packet,
assuming all is well. Some of this is checksum. In fact, I
recall some early statistics in which the checksum alg took up 40% of the
CPU cycles for processing incoming segments of TCO. I am lumping
IP level processing into TCP in the 1000-1200. This is absolutely
a WAG - so anyone with some hard data on instruction count is
most welcome to provide better info.

Roundtrip time and window size limitations affect performance
with respect to total throughput, all other things ignored.

Suppose you were operating at 10MB (megabytes)/second and
had a prop. delay of 10 microseconds on the ring.

With no constraints, and using 1000 byte packets, you would
be sending 10,000 packets/second to achieve full throughput.
That gives you 100 microseconds worth of insruction time.
At 14 MIPs, that is 1400 instructions. So, if you did nothing
else but TCP/IP, you might make it, with respect to
instruction rate.

Window size limitations: you can have 65,536 bytes outstanding
and at 10 MB/sec, it takes 6.5 ms to send them. Assuming you
can send the ack in the 100 microseconds you have to "process"
the packet, it takes 10 microseconds + 100 + 10 = 120 usec
(2 X propagation delay plus processing time) to get the ACK.
Even asusming another 100 usec to process the ACK, that 220 usec
worth of window needed, minimum. That's only 2200 bytes - so
even a factor of 10 increase is well within the window capacity
of the TCP.

The killer in all this is propagation and round-trip delay. It
doesn't take much to cause the window requirment for full
bandwidth to exceed the maximum allowed window of outstanding
bytes. For example, a round-trip time of 10 ms (the prop
delay of only 1000 miles, one way!) requires a window of 
over 100KB to maintain full capacity at 10MB/sec.

What these very sketchy and rough numbers suggest is that
window-based schemes won't be very satisfactory for long haul,
long delay, very high speed nets. Flow control based on rates is needed,
rather than on round-trip acks/permissions - of course, this is precisely
the kind of thinking that I believe underlies the work that
Dave Clark has been doing with NETBLT (Dave, holler if I
have put words/thoughts into your work inappropriately).

To sum up, TCP can probably be made to work well in low delay
systems at pretty high speeds, maybe up to 100 Mbit/sec, but
probably not at a gigabit and probably not at long haul cross
country at 100's of megabits/sec.

By the way, most of these same considerations apply to ISO TP,
in my estimation.

Vint