92

I want to create a docker-compose file that is able to run on different servers.

For that I have to be able to specify the host-ip or hostname of the server (where all the containers are running) in several places in the docker-compose.yml.

E.g. for a consul container where I want to define how the server can be found by fellow consul containers.

consul:
  image: progrium/consul
  command: -server -advertise 192.168.1.125 -bootstrap

I don't want to hardcode 192.168.1.125 obviously.

I could use env_file: to specify the hostname or ip and adopt it on every server, so I have that information in one place and use that in docker-compose.yml. But this can only be used to specifiy environment variables and not for the advertise parameter.

Is there a better solution?

Improve this question
1
  • This isn't something that swarm and compose handle out of the box at the moment. You could probably make this work on Weave, but you will need Powerstrip also, but that will take some more work. Commented Mar 16, 2015 at 11:52

7 Answers 7

Reset to default
23

docker-compose allows to use environment variables from the environment running the compose command.

See documentation at https://docs.docker.com/compose/compose-file/#variable-substitution

Assuming you can create a wrapper script, like @balver suggested, you can set an environment variable called EXTERNAL_IP that will include the value of $(docker-machine ip).

Example:

#!/bin/sh
export EXTERNAL_IP=$(docker-machine ip)
exec docker-compose $@

and

# docker-compose.yml
version: "2"
services:
  consul:
    image: consul
    environment:
      - "EXTERNAL_IP=${EXTERNAL_IP}"
    command: agent -server -advertise ${EXTERNAL_IP} -bootstrap

Unfortunately if you are using random port assignment, there is no way to add EXTERNAL_PORT, so the ports must be linked statically.

PS: Something very similar is enabled by default in HashiCorp Nomad, also includes mapped ports. Doc: https://www.nomadproject.io/docs/jobspec/interpreted.html#interpreted_env_vars

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

in default docker config there is a NO_PROXY environment variable that already contains docker-machine ip and it could be used in docker-compose.yaml
If you're running a docker swarm, you can get the node address with: docker system info -f "{{.Swarm.NodeAddr}}"
From inside a running container?
20

I've used docker internal network IP that seems to be static: 172.17.0.1

Improve this answer

2 Comments

That isn't visible to external services, they need the host public ip to connect into the container's exposed ports.
Worked for me as well. It's the IP of the default bridge network: docs.docker.com/v17.09/engine/userguide/networking/…. BUT - bear in mind there is better ways to solve this in production environments...! :-)
15

With the new version of Docker Compose (1.4.0) you should be able to do something like this:

docker-compose.yml

consul:
  image: progrium/consul
  command: -server -advertise HOSTIP -bootstrap

bash

$ sed -e "s/HOSTIP/${HOSTIP}/g" docker-compose.yml | docker-compose --file - up

This is thanks to the new feature:

  • Compose can now read YAML configuration from standard input, rather than from a file, by specifying - as the filename. This makes it easier to generate configuration dynamically:

    $ echo 'redis: {"image": "redis"}' | docker-compose --file - up
Improve this answer

2 Comments

To retrieve ip from docker-machine and execute as a one liner - sed -e "s/HOSTIP/$(docker-machine ip your-machine-name)/g" docker-compose.yml | docker-compose --file - up
From Linux only, you can sed -e "s/HOSTIP/$(hostname --ip-address)/g" docker-compose.yml | docker-compose --file - up
14

So, someone recently saw fit to downvote this answer. In all fairness, it was written almost 10 years ago. Move along and find something more current!

Is there a better solution?

Absolutely! You don't need the host ip at all for communication between containers. If you link containers in your docker-compose.yaml file, you will have access to a number of environment variables that you can use to discover the ip addresses of your services.

Consider, for example, a docker-compose configuration with two containers: one using consul, and one running some service that needs to talk to consul.

consul:
  image: progrium/consul
  command: -server -bootstrap
webserver:
  image: larsks/mini-httpd
  links:
    - consul

First, by starting consul with just -server -bootstrap, consul figures out it's own advertise address, for example:

consul_1    | ==> Consul agent running!
consul_1    |          Node name: 'f39ba7ef38ef'
consul_1    |         Datacenter: 'dc1'
consul_1    |             Server: true (bootstrap: true)
consul_1    |        Client Addr: 0.0.0.0 (HTTP: 8500, HTTPS: -1, DNS: 53, RPC: 8400)
consul_1    |       Cluster Addr: 172.17.0.4 (LAN: 8301, WAN: 8302)
consul_1    |     Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
consul_1    |              Atlas: <disabled>

In the webserver container, we find the following environment variables available to pid 1:

CONSUL_PORT=udp://172.17.0.4:53
CONSUL_PORT_8300_TCP_START=tcp://172.17.0.4:8300
CONSUL_PORT_8300_TCP_ADDR=172.17.0.4
CONSUL_PORT_8300_TCP_PROTO=tcp
CONSUL_PORT_8300_TCP_PORT_START=8300
CONSUL_PORT_8300_UDP_END=udp://172.17.0.4:8302
CONSUL_PORT_8300_UDP_PORT_END=8302
CONSUL_PORT_53_UDP=udp://172.17.0.4:53
CONSUL_PORT_53_UDP_ADDR=172.17.0.4
CONSUL_PORT_53_UDP_PORT=53
CONSUL_PORT_53_UDP_PROTO=udp
CONSUL_PORT_8300_TCP=tcp://172.17.0.4:8300
CONSUL_PORT_8300_TCP_PORT=8300
CONSUL_PORT_8301_TCP=tcp://172.17.0.4:8301
CONSUL_PORT_8301_TCP_ADDR=172.17.0.4
CONSUL_PORT_8301_TCP_PORT=8301
CONSUL_PORT_8301_TCP_PROTO=tcp
CONSUL_PORT_8301_UDP=udp://172.17.0.4:8301
CONSUL_PORT_8301_UDP_ADDR=172.17.0.4
CONSUL_PORT_8301_UDP_PORT=8301
CONSUL_PORT_8301_UDP_PROTO=udp
CONSUL_PORT_8302_TCP=tcp://172.17.0.4:8302
CONSUL_PORT_8302_TCP_ADDR=172.17.0.4
CONSUL_PORT_8302_TCP_PORT=8302
CONSUL_PORT_8302_TCP_PROTO=tcp
CONSUL_PORT_8302_UDP=udp://172.17.0.4:8302
CONSUL_PORT_8302_UDP_ADDR=172.17.0.4
CONSUL_PORT_8302_UDP_PORT=8302
CONSUL_PORT_8302_UDP_PROTO=udp
CONSUL_PORT_8400_TCP=tcp://172.17.0.4:8400
CONSUL_PORT_8400_TCP_ADDR=172.17.0.4
CONSUL_PORT_8400_TCP_PORT=8400
CONSUL_PORT_8400_TCP_PROTO=tcp
CONSUL_PORT_8500_TCP=tcp://172.17.0.4:8500
CONSUL_PORT_8500_TCP_ADDR=172.17.0.4
CONSUL_PORT_8500_TCP_PORT=8500
CONSUL_PORT_8500_TCP_PROTO=tcp

There is a set of variables for each port EXPOSEd by the consul image. For example, in that second image, we could interact with the consul REST API by connecting to:

http://${CONSUL_PORT_8500_TCP_ADDR}:8500/
Improve this answer

6 Comments

I didn't leave the -1 but from his using this for a command line option called --advertise I am guessing that the process is reporting it's address somewhere through some other sort of protocol to machines or clients not in docker. So it needs to know what it's external ip is (ie the docker host one) otherwise it is going to report the one docker assigns to it. So you post of "This is how linking works" completely misses the question since those ips are not visible outside the docker containers.
And how to use external_links or extra_hosts to connect from Docker to the machine host service? For example I got Postfix running on the host machine, but I want to connect from a Docker container to the Postfix service without starting another docker container...
A docker container can always contact services on the host using the address of the docker bridge. This will be the default gateway address inside the container.
This solution sorts out the example situation, but does not answer the question which is to provide the docker host IP to the container environment using docker-compose.
Yeah, this is old, and Docker behavior has changed. Docker compose creates user-defined networks for your services, so now you can simply refer to other containers in your compose file by name.
|
6

Environment variables, as suggested in the earlier solution, are created by Docker when containers are linked. But the env vars are not automatically updated if the container is restarted. So, it is not recommended to use environment variables in production.

Docker, in addition to creating the environment variables, also updates the host entries in /etc/hosts file. In fact, Docker documentation recommends using the host entries from etc/hosts instead of the environment variables.

Reference: https://docs.docker.com/userguide/dockerlinks/

Unlike host entries in the /etc/hosts file, IP addresses stored in the environment variables are not automatically updated if the source container is restarted. We recommend using the host entries in /etc/hosts to resolve the IP address of linked containers.

Improve this answer

2 Comments

this answer deserves more visibility
This answer is true, but then, it doesn't provide any solution to the question being asked -- how to use the host IP in docker-compose?
5

Create a script to set, every boot, your host IP in an environment variable.

sudo vi /etc/profile.d/docker-external-ip.sh

Then copy inside this code:

export EXTERNAL_IP=$(hostname -I | awk '{print $1}')

Now you can use it in your docker-compose.yml file:

version: '3'
services:
  my_service:
    container_name: my-service
    image: my-service:latest
    environment:
    - EXTERNAL_IP=${EXTERNAL_IP}
    extra_hosts:
    - my.external-server.net:${EXTERNAL_IP}

    ...
  • environment --> to set as system environment var in your docker container
  • extra_hosts --> to add these hosts to your docker container
Improve this answer

3 Comments

export EXTERNAL_IP=$(hostname -I | awk '{print $1}') thsi does not give external ip
To get the external you can use export EXTERNAL_IP=$(wget -qO- ifconfig.me | awk '{print $1}')
in 2024 still struggling on this
4

extra_hosts works, it's hard coded into docker-compose.yml but for my current static setup, at this moment that's all I need. 

version: '3'
services:
  my_service:
    container_name: my-service
    image: my-service:latest
    extra_hosts:
      - "myhostname:192.168.0.x"

    ...

    networks:
      - host
networks:
  host:
Improve this answer

2 Comments

I used your solution to make a dynamic one. I hope it fix to another people as your static one.
My latest setup includes a private local DNS Bind server.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.