K
Q

docker-compose scaling returns error 'address already in use'. How can I scale and maintain predictable / fixed IPs?

1/21/2020

I'm looking to find a way to scale my containers while still maintaining predictable IPs for each of them. For example I have the following docker-compose.yaml file with an arbitrary image. It has a fixed IP running on a defined network:

e.g.

version: '3'

services:
  my_postgres_db:
    image: postgres:9.6.16-alpine
    networks:
      static-network:
        ipv4_address: 172.20.128.2

networks:
  static-network:
    ipam:
      config:
        - subnet: 172.20.0.0/16

I run the following commands:

docker-compose up -d
docker-compose scale my_postgres_db=4

And I get the following errors:

Creating and starting root_my_postgres_db_2 ... error
Creating and starting root_my_postgres_db_3 ... error
Creating and starting root_my_postgres_db_4 ... error

ERROR: for root_my_postgres_db_2  Cannot start service my_postgres_db: Address already in use

ERROR: for root_my_postgres_db_3  Cannot start service my_postgres_db: Address already in use

ERROR: for root_my_postgres_db_4  Cannot start service my_postgres_db: Address already in use

This is understandable since I'm trying to bring up multiple containers that use the same fixed IP.

However, is there a way to scale-up containers while still knowing the IPs that they'd be running on? I can scale without a fixed IP, but I need to communicate with the container without manually checking what its IP is.

I'm starting to move more towards Kubernetes. I know you can have Deployments which contain multiple 'replicas' of containers, essentially scaling and grouping them into one concept. We can then communicate with the Deployment on one IP/Port using a Service.

I'm wondering if there's a way to do this using only Docker / Docker Compose?

-- solarflare
docker
docker-compose
kubernetes

Similar Questions

Is ClusterIP able to load balance traffic for multiple pods?
Prepare VPS for installing Kubernetes with Kubespray
How to reset grafana's admin password (installed by helm)
k3s - High Availability on raspberry pis - problems setting up
Kubernetes Custom metrics not available at Service level
Kubernetes ingress with backend serviceName regex/wildcard or selector

1 Answer

1/21/2020

I think the only way to achieve this only using docker-compose is creating separated services with fixed ip for each in your docker-compose files like:

version: '3'

services:
  my_postgres_db_1:
    image: postgres:9.6.16-alpine
    networks:
      static-network:
        ipv4_address: 172.20.128.2
  my_postgres_db_2:
    image: postgres:9.6.16-alpine
    networks:
      static-network:
        ipv4_address: 172.20.128.3
  my_postgres_db_3:
    image: postgres:9.6.16-alpine
    networks:
      static-network:
        ipv4_address: 172.20.128.4

networks:
  static-network:
    ipam:
      config:
        - subnet: 172.20.0.0/16

In kubernetes you can use use a best approach like Service or StatefulSet to do this.

StatefulSet is a great choice to achive what you want because it maintain the container spec:

Each Pod in a StatefulSet derives its hostname from the name of the StatefulSet and the ordinal of the Pod. The pattern for the constructed hostname is $(statefulset name)-$(ordinal).

So if you create a StatefulSet called 'my_postgres_db' with 3 replicas, your container will have every the same name (my_postgres_db-1, my_postgres_db-2 e my_postgres_db-3).

I hope it helps.

-- KoopaKiller
Source: StackOverflow