[Mep-dev] TCP Performance Issues

[Timothy J. Salo]

The issue of TCP performance over long-delay, high error-rate
paths keeps coming up in MEP/ACP discussions.  A few [long
and technical] comments on TCP performance might be in order.

Bob McGwier wrote:
> ...  Anyone who has ever tried to use streaming video over
> a cell phone or similar things which are just killed by the latency in
> trying to use traditional connections over traditional networks realizes
> that the network is not even coming close to its theoretical capacity and
> that it is the protocol itself which has caused the excessive backoff.

I assume that you are referring to poor TCP performance over
paths that have high error rates and/or long latencies.
Considerable work has been done on these topics over the last
20 years.

The fundamental issue is that TCP considers [almost] any
lost packet to be an indication of congestion in the network.
In order to avoid a congestive collapse of the network, TCP
[usually] backs off exponentially whenever it experiences
lost packets (because it assumes that the packets were lost
because of congestion in the network, not because of transmission
errors).  That is, TCP dramatically reduces the rate at which
it transmits when just a few packets are lost.  The result is
poor end-to-end performance.  (Every networking person should
read Van Jacobson's seminal 1988 SIGCOMM paper, "Congestion
Avoidance and Control".  Van pretty much singlehandedly saved
the Internet from congestive collapse.)

The result of this TCP behavior is that fairly low packet-loss
rates can dramatically reduce TCP's throughput.  (More
precisely, this results from the behavior of the Van Jacobson
Congestion Control, VJCC, algorithm used by TCP.)  This
problem becomes worse as latency increases. Matt Mathis
developed a nice model of this behavior:

   Mathis, Matt, J. Semke, J. Mahdavi, T. Ott, "The
   Macroscopic Behavior of the TCP Congestion Avoidance
   Algorithm", Computer Communication Review, volume 27,
   number3, July 1997

Latency, in and of itself and in the absence of errors, is
pretty much _not_ an issue for TCP.  People have made
TCP run at full speed over 622 Mbps GEO satellite links.
(See Mark Allman's work with the ACTS High Data Rate
terminal.)  All you need is a modern TCP implementation
(with the window-scale option and a few other pieces),
perhaps some tuning of the end hosts, and an end-to-end
path with a very low error rate.  Google "TCP satellite"
for over two million Web pages on this topic.

This leads to the question: How can you make TCP perform
well over lossy paths that may also have high latencies?

The first answer is to reduce the packet-loss loss rate on
the path, or more precisely, on each link in the path.
See, for example, RFC 3819, "Advice for Internet
Subnetwork Designers" (edited by Phil Karn).

Another approach is to use a performance-enhancing proxy
(PEP).  PEPs essentially break the end-to-end TCP connection
into several concatenated connections.  The connection that
runs over the lossy link assumes that lost packets result
from transmission errors, rather than from network congestion.
That is, this connection does not run VJCC, does not assume
that lost packets indicate network congestion, and does not
back off exponentially when it sees lost packets.  You might
look at "A Comparison of Mechanisms for Improving TCP
Performance over Wireless Links" by Balakrishnan et al.

   http://research.microsoft.com/~padmanab/papers/ton97.pdf

Or, Google "TCP performance" (half-a-million hits).

> The problem will be exacerbated in geostationary orbit with latencies that
> are much longer than traditional 3G data networking.

To restate your claim more precisely: the effect of packet
losses on TCP performance increases with latency.

See Matt Mathis' model, referenced above.

The MEP won't have these long latencies, but I hope that
the knowledge gained from the MEP project will be useful
to an ACP or other satellite-based project.

- - - -

Note that this e-mail is only a terse primer on TCP performance.
I will talk about my views on the DTN protocols and the MEP
project in a future e-mail message.

-tjs