Docker Compose - Share named volume between multiple containers

Question

I'm using docker-compose and v3. I'm trying to mount a volume in docker:

./appdata:/appdata

I'd like to have this as a volume and then reference that volume from multiple containers. The volume configuration reference only shows data-volume: as a named volume, with no value, so it doesn't look like the above.

services:

    nginx:
        build: ./nginx/
        ports:
            - 80:80
        links:
            - php
        volumes:
            - app-volume

    php:
        build: ./php/
        expose:
            - 9000
        volumes:
            - app-volume

volumes:
     app-volume: ./appdata:/appdata

This gives me:

ERROR: In file './docker-compose.yml', volume 'app-volume' must be a mapping not a string.

Obviously I know I need to change the volumes key/value pair, but I'm not sure how to change this so I can share a volume between services.

I've also checked out volumes_from but this effectively just allows inheritance from other containers. I've seen someone use volumes_from on another container that contains the mapping they want, but with command: true set so that the container is never actually run, which to me just seems like a hack.

How can I do this?

---

Note, I do have the following working:

nginx:
    volumes:
        - ./appdata:/appdata
php:
    volumes:
        - ./appdata:/appdata

But that's just duplication and is something I'm hoping a named volume could help me avoid :-)

Toby

Answer 1

You can use one of the two options:

1. Named volumes: https://docs.docker.com/compose/compose-file/07-volumes/

2. extension-fields to avoid duplicating volumes source and prevent yourself from future typos:

version: '3.5'

x-services-volume:
  &services-volume
  type: bind
  source: ./appdata
  target: /appdata

services:

    nginx:
        build: ./nginx/
        ports:
            - 80:80
        links:
            - php
        volumes: *services-volume

    php:
        build: ./php/
        expose:
            - 9000
        # Use same way as for nginx if target override not needed.
        volumes:
            - <<: *services-volume
            target: /opt/target-override

NOTE: This feature is available starting from version 3.4 file format.

Lynnett

If the *services-volume is just a pointer to the value set above, this looks awesome... I'll have to try it. – Toby

@Toby yes, it is, also note that docker-compose file version shall be 3.4+ – Lynnett

Named volumes, aka the top-level volumes field, seem to still be a thing in v3 of docker-compose. – Mona

I tried exactly this approach to share data between three containers and a host. In my case, the folder structures in the containers are equal. Seems as if this works as expected. What I don't get is, which container (or the host) has the privileged right when starting it up again? What I mean: When issuing a docker-compose down, each container's internal file-system is saved somewhere internally. When I now change data on the host-folder and do a docker-compose up again: What is the rule for the content in the shared folder? Will data from the host overwrite the internally saved data?thx! – Yt

@Yt it's retained and same for all containers, so you will have content retained. – Lynnett

@AndriyIvaneyko but thats exactly my question: what are the retaining rules. Again: If I have a shared volume and I delete a file on the host while docker is not running and then I start it up again with the docker compose file: will the file be deleted for all containers or will the containers take their information from their retained state and recreate the file on the host system. Is the logic in precedence somewhere specified? – Yt

@Yt the folder is shared, imagine google drive and real users + shared folder, if someone delete file on that folder than it's gone for the rest of users – Lynnett

@AndriyIvaneyko Are named volumes actually removed? Couldn't find any references on that. – Luck

Named values are not removed, at least as of May 2023, check docs.docker.com/compose/compose-file/07-volumes – Timehonored

kevinVanRyckegem @devrimbaris good point, I've updated answer, thank you folks. – Lynnett

Answer 2

The named volumes can be shared across containers in the following way:

services:
    nginx:
        build: ./nginx/
        ports:
            - 80:80
        links:
            - php
        volumes:
            - app-volume:location_in_the_container

    php:
        build: ./php/
        expose:
            - 9000
        volumes:
            - app-volume:location_in_the_container

volumes:
     app-volume: 

Here's an example config that I use for better understanding. I'm exposing the static files generated from my web container to a named volume called static-content which is then read and served by the nginx container:

services:
  nginx:
    container_name: nginx
    build: ./nginx/

    volumes:
      - static-content:/usr/src/app

  web:
    container_name: web
    env_file: .env
    volumes:
      - static-content:/usr/src/app/public
    environment:
      - NODE_ENV=production

    command: npm run package

volumes:
  static-content:

Stridulate

But there are two locations I use in my example, a host one and a container one. Originally it looked like this: - ./appdata:/appdata but with your solution, I can only see one of these values? – Toby

Both the containers can write to the volume AFAIK. so if you expose a folder from one of your containers into the named volume - it can be seen and overwritten by the other container and vice versa. If its still unclear - i'll paste my example code to try explaining it better – Stridulate

Where do you set the location of static_content on the host filesystem? – Hus

Whitespace in app-volume: location_in_the_container is wrong. – Fowlkes

What if /usr/src/app in the nginx container and /usr/src/app/public in the web container both got original content, which one will be used and why? – Secularity

you're right. The problem is that i completely lost context of this answer as its been quite a while. Regarding your question - its a named volume - i believe the mapping from host-> docker will be done internally by docker @TravisBear - i belive the location (mapping to a local drive) is managed by docker internally. regarding whose content is used, i believe both can write to the volume - so its upto you to ensure theres no overwriting by files. In my case , only node writes to the volume while nginx serves the file – Stridulate

Thanks for this answer, yet even having pretty much the same docker-compose.yaml as you, I get a Cannot create container for service mobile-app: invalid volume specification: '..._appdata::rw': invalid mount config for type "volume": field Target must not be empty I have checked the official [documentation]("docs.docker.com/compose/compose-file/…) and there doesn't seen to be a difference. – Braley

@TravisBear for this use case (sharing data between containers) there is no really need to have it on the host. Example with static data is great - you execute collectstatic in one container and want results to be available in another one, but you don't care about host folder – Obau

@Kannaj TravisBear's question is the one that correctly identifies the issue I find the most confusing. How in the compose file can you specify where the named volume is sourced? I don't want to leave it up to the docker engine to determine where to store the named volume on the host, I want to specify a path. – Manuscript

@BenCollins - I believe you can use path based volumes for your use case - nickjanetakis.com/blog/… – Stridulate

I believe that "path based volumes" is actually referring to "bind mounts" - which are much more easy to google for. – Toby

@TravisBear Looks like the new answer from Andriy Ivaneyko below might do this. – Toby

Does this work for 3+ docker compose schema? For me, directory is always empty, even though another container has data in it. – Poe

Answer 3

This solves it without using named volumes:

      volumes:
          - ./appdata:/appdata

So, it looks like:

services:

  nginx:
      build: ./nginx/
      ports:
          - 80:80
      links:
          - php
      volumes:
          - ./appdata:/appdata

  php:
      build: ./php/
      expose:
          - 9000
      volumes:
          - ./appdata:/appdata

Adversary

Ah, nice timing! I did this above (see my change). However, it seems we're still duplicating the mapping. If I use this over 3 containers, it gets big. Can we use named containers to avoid this duplication? – Toby

The thing is that named volumes is not something only about syntax and clear code. It will create a volume inside docker data installation directory and you will not have your local files there (the ./appdata). Is it useful for you anyway? – Adversary

I definitely need the ./appdata, that is what I'm trying to do. Leave this answer here though :) +1 – Toby

What happens if I have two containers of the same image, uploading file (via upload file service) in one container, it's gonna be available in the other one? if not, how can I do that? – Termination

Answer 4

Previous answers helped me a lot (thanks!) but it took some more time to figure out how to set options to create a shared tmpfs volume (memory file system). I'm sharing it in the hope to make life easier for developers in the same situation.

An example showing only the relevant parts of docker-compose.yml:

version: '3'

volumes:
  shared-tmpfs:
    driver: local
    driver_opts:
      type: "tmpfs"
      device: "tmpfs"
      o: "size=256m,uid=1000"

services:
  nginx:
    volumes:
      - shared-tmpfs:/tmp/mytmpfs

  php-fpm:
    volumes:
      - shared-tmpfs:/tmp/mytmpfs

I use it to spare my SSD from heavy writes (building lot's of static html files) in development/watch mode.

You can read more about driver_opts in the official Docker docs here.

Sussi

What about the accepted answer didn't work for you? Here you have to define it twice - with the accepted answer's & and * notation so you only have to define it once? – Toby

@Toby I do like the accepted answer but I wasn't able to achieve the same with that approach. That solution binds an existing directory, my solution creates a new tmpfs volume on-the-fly with the given options. It may be my fault, but I wasn't able to convert it to the form of the accepted answer (extension fields). – Sussi