46

I am using docker compose version 3.3 and want to use environment variable to define the volume name. I looked at a related question, but that seems to be quite old. With long syntax supported in 3.2, is there a way to achieve that? Here is what I tried in my docker compose file:

version: '3.3'
services:
  target:
    image: "my-registry/my-image:${IMAGE_TAG}"
    volumes:
        - type: volume
          source: ${VOLUME_NAME}
          target: /data
    ports:
     - "${TOMCAT_PORT}:8080"

volumes:
  ${VOLUME_NAME}:

Obviously this syntax does not work as volume name is not substituted in the keys and throws the following error:

volumes value Additional properties are not allowed ('${VOLUME_NAME}' was unexpected)

Any help will be much appreciated.

Improve this question

5 Answers 5

Reset to default
34

If you want to avoid conflicts between different projects, you just need to specify --project-name "your_project_name" for docker-compose. It adds namespace for all services and volumes.

For example, you may use the same docker-compose.yml for few projects:

volumes:
    dbdata_mysql:

services:
    mysql_db:
        image: mysql:5.7
        volumes:
            - "dbdata_mysql:/var/lib/mysql"

If you start your projects with:

docker-compose --project-name "first_project" up -d
docker-compose --project-name "second_project" up -d

it will create namespaced volumes:

$ docker volume ls | grep dbdata_mysql

local     first_project_dbdata_mysql
local     second_project_dbdata_mysql
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

THAT's the answer! please upvote
Awesome find! I also discovered that you can set an environment variable COMPOSE_PROJECT_NAME to override the project name. And I suspect you can probably put that in a .env file for a more persistent setting.
How to do it with docker stack deploy?
Note that this is already the default behaviour. If you don't specify the --project-name argument, docker-compose uses the directory to determine project name and prefixes all volumes and containers with that name.
25

This is expected behavior - Compose only does variable interpolation in values, not keys. See here.

In my project I use external structure:

version: '3.1'
services:
### Code from branch develop ###
  applications:
    image: registry.gitlab.lc:5000/develop/ed/develop.sources:latest
    volumes:
      - developcode:/var/www/develop
    deploy:
      replicas: 1
      update_config:
        parallelism: 1
        delay: 5s
      restart_policy:
        condition: on-failure
      placement:
        constraints: [node.role == manager]
### PHP-FPM ###
  php-fpm:
    image: registry.gitlab.lc:5000/develop/ed/php-fpm-ed-sq:latest
    volumes:
      - developcode:/var/www/develop
    expose:
      - "9000"
    deploy:
      replicas: 2
      update_config:
        parallelism: 1
        delay: 5s
      restart_policy:
        condition: on-failure
      placement:
        constraints: [node.role == manager]
    logging:
      driver: gelf
      options:
        gelf-address: "udp://${GRAYLOG_ADDR}:12201"
        tag: "php-fpm"
### Nginx ###
  nginx:
    image: registry.gitlab.lc:5000/develop/ed/nginx-ed-sq:staging
    volumes:
      - developcode:/var/www/develop
    ports:
      - "80:80"
      - "443:443"
    deploy:
      replicas: 2
      update_config:
        parallelism: 1
        delay: 5s
      restart_policy:
        condition: on-failure
      placement:
        constraints: [node.role == manager]
### Volumes Setup ###
volumes:
  developcode:
    external:
      name: code-${VER}

but first of all I need create external volume manually, e. g.:

export VER=1.1 && docker volume create --name code-$VER

You can see created volume:

docker volume ls
DRIVER              VOLUME NAME
local               code-1.0
local               code-1.1

And after that, deploy services using:

env $(cat .env | grep ^[A-Z] | xargs) docker stack deploy --with-registry-auth --compose-file docker-compose.yml MY_STACK
Improve this answer

1 Comment

Note in newer versions: "the external.name property is deprecated in favor of simply using the name property."
13

This is how I got it done for my case inside .env

# To use a code volume create the volume mapping an existing folder like this and pass the value of <volume_name> below
# docker volume create --driver local --opt type=none --opt device=<full_path_to_folder> --opt o=bind <volume_name>
PROJECT_VOLUME_CODE=myproject_code

then inside docker-compose.yml (version 3.7) or higher Inside the service

    volumes:
      - type: volume
        source: code
        target: /var/www/html

Outside

volumes:
  code:
    external: true
    name: ${PROJECT_VOLUME_CODE}
Improve this answer

2 Comments

This is the correct way, and it works without using .env files too
I get external volume not found when doing this in 3.8 on ubuntu. Did anything change between 3.7 and 3.8?
12

I managed to use environment variables with this syntax:

volumes:
    - "$HOME/.cache/composer:/tmp/composer_cache"
Improve this answer

1 Comment

This is golden! It will take the value from a .env file and if you want to use a default value in case there is nothing set in the .env file this is the format - "${PGVECTOR_PATH:-./localdev/postgres_pgvector}:/var/lib/postgresql/data/" Using this we can all share the same docker compose file and adjust the path locations at a user level without it getting checked into the code base as .env files should be added to gitingore.
3

If you need to change complex stuff between environments, like using completely different volume settings, you should override your configuration with multiple docker-compose files.

Environment variables should be used for simple values only.

Using multiple configuration files lets you define a default docker-compose.yml file with your base configuration, and another docker-compose.override.yml for the changes you need for a particular environment.

Then, when creating the services, docker compose will merge the configuration files.

In your case, your default configuration might look like this:

# docker-compose.yml
version: '3.3'
services:
  target:
    image: "my-registry/my-image:${IMAGE_TAG}"
    volumes:
        - type: volume
          source: vprod
          target: /data
    ports:
     - "80:8080"

volumes:
  vprod:

And your dev override file may look like this:

# docker-compose.override.yml
services:
  target:
    volumes:
        - source: vdev
          target: /data
    ports:
     - "8080:8080"

volumes:
  vdev:

Notice that not all services and not all keys need to be repeated in the override file.

When you run docker-compose up both configurations will be merged with the override file taking precedence.

Docker compose picks up docker-compose.yml and docker-compose.override.yml by default, if you have more files, or files with different names, you need to specify them in order:

docker-compose -f docker-compose.yml -f docker-compose.custon.yml -f docker-compose.dev.yml up -d
Improve this answer

1 Comment

I think this is the more relevant link to the docs, because the one that you linked is talking about extending compose files, not overriding/ merging configurations

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.