Load Balancing with Nginx

Years ago I needed extra modules or separate tools to load balance WebSockets and TCP connections. Today, open-source nginx handles all of it out of the box.

HTTP load balancing

http {
    upstream app_backend {
        least_conn;                       # send requests to the least busy server
        server 10.0.0.11:8080 weight=2;
        server 10.0.0.12:8080;
        server 10.0.0.13:8080 max_fails=3 fail_timeout=30s;
        server 10.0.0.14:8080 backup;     # only used if the others are down
    }

    server {
        listen 80;
        server_name app.example.com;

        location / {
            proxy_pass http://app_backend;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}

Balancing methods:

• Round robin (default) – requests are distributed in turn, optionally weighted with weight.
• least_conn – the server with the fewest active connections gets the next request.
• ip_hash – clients stick to the same server, useful for apps with local sessions.
• hash $key consistent – stickiness based on any value, e.g. a cookie or URL.

Health checks: Open-source nginx uses passive health checks. If a server fails max_fails times within fail_timeout, it's taken out of rotation for that time. Active health checks are part of NGINX Plus.

WebSockets

WebSockets need HTTP/1.1 and the Upgrade headers to be passed through:

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    listen 80;
    server_name ws.example.com;

    location /socket/ {
        proxy_pass http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_read_timeout 3600s;   # keep idle connections open
    }
}

The map block belongs in the http context.

TCP and UDP load balancing

For non-HTTP traffic such as databases or message brokers, use the stream module. The stream block sits at the top level of nginx.conf, next to http – not inside it:

stream {
    upstream postgres_replicas {
        server 10.0.0.21:5432;
        server 10.0.0.22:5432;
    }

    server {
        listen 5432;
        proxy_pass postgres_replicas;
        proxy_connect_timeout 5s;
    }
}

On some distributions the stream module is a separate package (e.g. libnginx-mod-stream on Debian/Ubuntu). Check with nginx -V.