Router Queues Visualization | Bufferbloat, Tail Drop, RED, CoDel

The idea: a full buffer is delay, not throughput

A router forwards packets onto each outgoing link one at a time. When packets arrive faster than the link can send them, the extra ones wait in the link's output queue. Every packet in front of you adds its transmission time to your wait. When the queue is full, the router must drop something; the rule that decides what to drop, and when, is the queue discipline.

A home LAN at 1 Gbit/s sends bursts into the router's output queue, which drains onto a 12 Mbit/s uplink at one packet per millisecond. A new packet's wait is the number of packets ahead divided by the link rate: 11 packets ahead is 11 ms, 100 packets ahead is 100 ms. The round trip is 20 ms on the wires plus the wait in the queue — The queue sits where a fast network meets a slow link; each packet ahead of you costs one transmission time.

The page models a home router with a 12 Mbit/s uplink: one 1500-byte packet per millisecond. A laptop uploads a big file with TCP Reno, and the base round trip is 20 ms, so 20 packets in flight are enough to keep the uplink busy (the bandwidth-delay product, BDP). A phone sends pings through the same queue. One tick is 1 ms.

Why TCP fills any buffer you give it

TCP Reno keeps raising its congestion window cwnd by one packet per round trip until a packet is lost, then halves it. Once cwnd is above the 20 packets the link needs, every extra packet does not go faster; it just waits in the queue. With tail drop the only loss signal comes when the buffer is full, so the queue stays long all the time: a standing queue. See TCP congestion control for the sender's side.

Bufferbloat

With a 100-packet buffer and tail drop, the queue climbs from about 40 to 99 packets in 6 seconds and never empties. The uplink is 100% busy, so the upload is as fast as it can be, but every packet now waits about 96 ms. A ping sent at 5.5 s waits 97 ms and takes 117 ms instead of 20 ms. That extra delay hits everything sharing the link: video calls, games, DNS lookups, web pages. Too much buffering in front of a slow link is called bufferbloat. (Demo 1: big buffer + tail drop = bufferbloat.)

Making the buffer small helps the delay: with 10 packets the wait stays under 10 ms. But 10 packets is less than the BDP, so after each halving cwnd drops below 20 and the uplink goes idle for a moment: 96% busy here. A fixed packet count is also wrong as soon as the link rate changes: 10 packets is 1 ms on a 120 Mbit/s link and 100 ms on a 1.2 Mbit/s one. (Demo 2: small buffer: short wait, link sometimes idle.)

Active queue management: drop before the buffer is full

Active queue management (AQM) drops (or ECN-marks) a few packets early, while the buffer still has room, so TCP slows down before a standing queue forms. The buffer stays large enough to absorb short bursts.

Three panels. Tail drop: drop chance is zero until the queue is full, then 100 percent. RED: drop chance is zero below an average queue of 5 packets, rises to 10 percent at 15, then jumps to 100 percent. CoDel: the wait of the head packet over time; once it has stayed above the 5 ms target for 100 ms, packets are dropped, closer and closer together, until the wait falls below 5 ms — Tail drop looks at free space, RED at the average queue length, CoDel at how long packets actually waited.

Discipline	What it measures	When it drops	Where
Tail drop	free space in the buffer	only when the buffer is full	arriving packet
RED	average queue length (w_q = 0.002)	at random: 0 below min_th = 5, up to max_p = 10% at max_th = 15, always above	arriving packet
CoDel	sojourn time: how long each packet waited	when the wait stays above target = 5 ms for interval = 100 ms; then the next drop after 100/√count ms	head of the queue

RED: random early detection

RED (Floyd and Jacobson, 1993) keeps an exponentially weighted average queue length and drops each arriving packet with a probability that grows with that average. Random drops hit the flows that send the most, and different flows do not all halve at the same moment. Once settled, the queue in the page stays short (at most 13 ms) and the uplink is 88% to 98% busy in each half second. (Demo 3: RED drops early by the average queue.)

The first second shows RED's weak point. The average moves slowly, so during slow start the real queue reaches 80 packets before RED reacts; then the average is above max_th and RED drops almost every packet while the real queue is already empty, and from 0.5 s to 1 s the uplink is only 22% busy. The right min_th, max_th, max_p and w_q depend on the link rate and the traffic, which is why RED was hard to tune and was rarely turned on.

CoDel: controlled delay

CoDel (Nichols and Jacobson, 2012; RFC 8289) does not count packets at all. It stamps each packet with its arrival time, and when a packet leaves it looks at the sojourn time, how long the packet waited. A short burst makes some packets wait, but the queue empties again; a standing queue means even the shortest wait stays high. So CoDel acts only when the wait has stayed above target = 5 ms for a whole interval = 100 ms (about one round trip). Then it drops the packet at the head and enters the dropping state, dropping again after interval/√count: 100 ms, 71 ms, 58 ms, 50 ms, … until the wait falls below the target.

Because it measures time, the same two numbers work on any link rate, with no tuning. In the page the queue wait averages 3 ms (8 ms at most) with the uplink 96% busy, and a ping takes 22 ms instead of 117. (Demo 4: CoDel drops packets that waited > 5 ms.) Dropping at the head also tells the sender sooner than dropping at the tail, because the lost packet's gap shows up at the receiver without waiting through the queue.

Switching a bloated queue to CoDel shows the control law at work: the queue of about 75 ms drains to under 10 ms within a few hundred milliseconds, while the upload keeps the link full. (Demo 5: switch tail drop to CoDel mid-run.)

The five runs side by side

Buffer, discipline	queue wait, last 0.5 s	uplink busy	ping round trip
100 packets, tail drop	96 ms average	100%	117 ms
10 packets, tail drop	3 ms average, 9 at most	96%	20 ms
100 packets, RED	6 ms average, 12 at most	97% (88% to 98% after 1 s)	21 ms
100 packets, CoDel	3 ms average, 8 at most	96%	22 ms
tail drop, then CoDel at 3 s	4 ms average, 9 at most	97%	27 ms

One TCP flow on a short path is the hardest case for any short queue; with many flows their halvings overlap less and the link stays fuller.

In practice

Linux ships tail drop (pfifo), red, codel and fq_codel; most distributions use fq_codel as the default qdisc (tc qdisc show). fq_codel gives each flow its own queue with CoDel, so a ping never waits behind an upload at all.
The queue that matters is the one in front of the slowest link, often the home router's uplink or the cable/DSL modem. Home routers fix it with "SQM": shape traffic to slightly below the line rate so the queue forms in the router, where fq_codel or CAKE manage it.
A quick test: run a big upload and a ping at the same time. If the ping time jumps from 20 ms to hundreds of milliseconds, the path has bufferbloat.

What the page leaves out

Several competing flows and fair queueing (fq_codel, CAKE); ECN marking instead of dropping; retransmissions, fast recovery and timeouts (a lost packet simply counts as gone and cwnd halves at most once per round trip); RED's gentle mode, adaptive RED and PIE; the download direction, Wi-Fi aggregation and the queues inside modems and drivers (BQL); packets of different sizes, so the page counts packets instead of bytes.