What is bbr?

BBR means Congestion-Based Congestion Control. Its a TCP congestion control mechanism, interpreting packet loss as “congestion”. There are different TCP congestion control mechanism available and Google BBR is one of them. RENO and CUBIC are few other TCP congestion control mechanism available.

The TCP congestion control mechanism like RENO and CUBIC are loss-based congestion control mechanism. Using it will cause low throughput when bottleneck buffers are small and when bottleneck buffers are large it causes buffer-bloat.

For Fixing these problems, a TCP delay-controlled TCP flow control algorithm from Google, called BBR implemented. Google BBR efficiently control the buffer-bloat and low throughput by continuously estimate the flow’s RTT and the flow’s sustainable bottleneck capacity.

Google Cloud Platform, GCP uses BBR and allows Applications to load lightning speed over the public network. Google services like YouTube uses this BBR and users will get better experience from YouTube.

Benefits of Google BBR

Higher throughput

Lower latency

Now suppose there is live website in BBR enabled server then the website content is sent to users browsers using BBR. This means faster webpage downloads for users of your site.
There is also a bbr github repository available with scripts, and other materials related to the BBR congestion control algorithm.

How we can Install or Enable Google BBR ?

Currently Google BBR can only be enabled in Linux operating systems. Suppose we have Linux machine is running kernel 4.9 or newer, we can enable BBR. By default the Linux servers uses Reno and CUBIC but the latest version kernels also includes the google BBR algorithms and can be enabled manually.

So a normal question arise is does windows support BBR? If not what is the TCP congestion control mechanism used by a windows servers ?. So lets deviate a little and see whats happening in a windows server. Use below command to show the enabled TCP congestion control mechanism in a Windows server.

Copy to Clipboard

Take a look at the “Add-On Congresion Control” parameter.

Copy to Clipboard

none: Use the built-in standard congestion control algorithm.
ctcp : Use the add-on Compound TCP congestion control algorithm.

The powershell command is

Copy to Clipboard
Copy to Clipboard

Okay, as per the results, the windows server 2016 and 2012 uses the default congestion control algorithm as DCTCP. Another one supported is CTCP.

Below is definition of both congestion control algorithm as per Microsoft docs.

CTCP. Compound TCP increases the receive window and amount of data sent. This value can improve throughput on higher latency connections.
DCTCP. Data Centor TCP adjusts the TCP window based on network congestion feedback based on Explicit Congestion Notification (ECN) signalling. This value can improve throughput on low latency links.

From windows server 2019 onward the default one become CUBIC, like Linux.

Now lets see how Google BBR can be enabled in Linux.

On centos 8

Issue below command to know the current kernel.

Copy to Clipboard

Issue Below commands to know the current tcp congestion control

Copy to Clipboard

Out of the box, Linux uses Reno and CUBIC and the default one is cubic

In case of centos 8, we can see the default kernel is 4.18 which is already higher than the needed 4.9.

So add below lines in /etc/sysctl.conf and issue command sysctl -p

Copy to Clipboard

Issue below commands to verify.

Copy to Clipboard

Now lets take a look how it can be done in centos 7

Verify the kernel using below command.

Copy to Clipboard

Import the ELRepo GPG Key:

Copy to Clipboard

Install the ELRepo repository:

Copy to Clipboard

Install the latest stable kernel using the ELRepo repository:

Copy to Clipboard

Check latest stable kernel installation

Copy to Clipboard

Show all entries in the grub2 menu:

Copy to Clipboard

Indexing starts at 0. This means that the 5.7 ( which is the latest one installed) kernel is located at 1:

set the default kernel

Copy to Clipboard

reboot the server.

Copy to Clipboard

After reboot, verify the loaded kernel.

Copy to Clipboard

Enable BBR

Copy to Clipboard

verify bbr

Copy to Clipboard

On Ubuntu 18.04, the kernel version is greater than 4.9. So just need to add sysctl entry, like we did it earlier and that will enable Google bbr on Ubuntu 18.04

Copy to Clipboard

For Ubuntu 16 follow below steps.

Verify the current kernel version

Copy to Clipboard

Install Hardware Enablement Stack (HWE) to update kernel automatically:

Copy to Clipboard

reboot the server and issue below command for enabling bbr

Copy to Clipboard

For enabling it in Ubuntu 14 or Debian 8 follow below steps.

Download a latest kernel from repo ulr https://kernel.ubuntu.com/~kernel-ppa/mainline/

Copy to Clipboard

After that enable BBR as usual.

Copy to Clipboard

Google BBR testing with iperf

The iperf is a commonly used network performance measurement tool available in Linux. It measures the throughput of the network. lets use the command for testing the network Performance.

In the server issue below command. If the command is not found install it using apt-get.

Copy to Clipboard

After that issue below command in your client PC.

Copy to Clipboard

Below is the test results before enabling Google BBR

Copy to Clipboard

Test Results After BBR enabled.

Copy to Clipboard

BBR congestion control PRE and POST average stats collected for 30 seconds using the iperf command is

Pre results    : BBR 281 MBytes 76.7 Mbits/sec
POST result : BBR 283 MBytes 79.0 Mbits/sec

In our case there wasn’t any high change in the speed but showed a difference. Feel free to test it on your servers. This concluded the BBR setup and testing in a Linux Box. Post your comments in the comment box provided below.