Load Balancing with Docker Compose and Nginx doesn't distribute traffic across replicas

Question

I'm using Docker Compose to run a simple HTTP server on multiple replicas to test Load Balancing capabilities of Docker Compose and Nginx.

There are 2 services in the docker-compose.yml file:

services:

  server:
    image: python:3.12-alpine3.19
    command: python -m http.server 8000
    deploy:
      replicas: 2

  nginx:
    image: nginx:1.23.4-alpine
    ports:
      - 80:80
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf

The configuration file for Nginx called nginx.conf in the current working directory has the following configuration:

upstream server {
  server server:8000;
}

server {
  listen 80;

  location / {
    proxy_pass http://server;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
  }
}

After running the Docker-Compose services using docker compose up command, I can successfully reach the HTTP server like that:

curl -XGET http://localhost/

However, if I terminate the replica of the server service that handled the initial request, the next requests to the HTTP server ends up with a timeout error, without reaching the available replica. The following error is returned by Nginx:

nginx-1  | 2025/09/30 12:50:16 [error] 35#35: *11 upstream timed out (110: Operation 

timed out) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /

HTTP/1.1", upstream: "http://172.18.0.2:8000/", host: "localhost"

The problem is that Nginx didn't route the traffic to the available replica of the server service to handle the requests.

The question is, what might be a problem in the current scenario and what is the right way to fix it ?

System Configuration:

OS: Ubuntu 24.04
Docker version: 28.0.2
Docker Compose version: 2.39.4

Lorenzo Sibi · Accepted Answer · 2025-10-21 13:41:48Z

1

Here your nginx configuration only resolves server:8000 to a single IP address at startup, not to all replicas. When that container dies, Nginx keeps trying the cached IP instead of discovering the other replica soyou need to enable DNS reresolution in Nginx. Here's my example example for a correct nginx.conf:

upstream server {
  server server:8000 max_fails=3 fail_timeout=30s;
  
  # Force DNS re-resolution
  keepalive 16;
}

server {
  listen 80;

  location / {
    # Use a variable to force DNS resolution on each request
    set $backend "server:8000";
    proxy_pass http://$backend;
    
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    
    # Important timeout settings
    proxy_connect_timeout 5s;
    proxy_next_upstream error timeout http_502 http_503 http_504;
  }
}

Let me know if it works :)

answered Oct 21 at 13:41

Lorenzo Sibi

1565 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Joe Doe Oct 25 at 22:06

Hi, @lorenzo-sibi. Many thanks for your investigation, and introducing the concept of DNS resolution in Nginx. It's highly appreciated. Unfortunately, after changing the nginx.conf file according to your recommendations, it still produces the same timeout error. It also worth mentioning that there was a need to slightly modify your configuration file for Nginx, by removing the port number from the value of $backend variable, to avoid 502 Bad Gateway Error. Anyway, it seems that Docker Compose doesn't provide Load Balancing capabilities unlike Docker Swarm...

Collectives™ on Stack Overflow

Load Balancing with Docker Compose and Nginx doesn't distribute traffic across replicas

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related