Mount docker Host path to Container

Question

I have a question concerning Docker. I'm running a Docker:dind Container, which I'm controlling throuh the API. I'm calling the Dind Container with the following Command

docker run --network some_network --prvileged -e DOCKER_TLS_CERTDIR= -d -p 2375:2375 -p 2376:2376 -v TransformationData:/mnt docker:dind

through the named volume, my Data is getting inside the Dind Container in the /mnt Directory.

Now it gets tricky. I want to run a Container through the API which transforms the Data in the /mnt directory and saves the transformed Data in the same directory, so I can use it through the named volume. the following are the Calls for the API

<REQUEST OPERATION="POST" URL="/containers/create" CONTENT-TYPE="application/json">
     <CONTENT>
        {
            "Image":        "citygml4j/citygml-tools",
            "AttachStdin":  true,
            "HostConfig": {"Binds": [
                            "/mnt:/data[[r]shared, [rw]]"
                    ]}} ,
            "Cmd":[
            "to-cityjson",
            "/data"
            ]            
        }
     </CONTENT>
  </REQUEST>

<REQUEST OPERATION="POST" CONTENT-TYPE="application/json">
        <xsl:attribute name="URL">/containers/<xsl:value-of select="substring(//CONTENT,11, 7)" />/start</xsl:attribute>
        <CONTENT>{}</CONTENT>
    </REQUEST>

The Container gets created, runs and exits, but the Data inside the /mnt Directory stays the same. instead of the "bind" Config I also tried a "mount" config with the same result.

One possible Cause for this Problem is, that the citygml-tools Container does not have write Permission on the /mnt directory of the host. I tried to fix that with the privileged Tag, but that did not work. Also the [[r]shared, [rw]] part in the "bind" should permit write, but it does not work either.

The annoying Part about this is, that the citygml-tools Container does not give any Error Messages or Lo files, so I don't actually know what's happening while I try. It's just a Trial and error like thing right now...

Long story short: What is the best Practice for Accessing Host Data from a Container, and write Data in the Host filesystem from a Container?

Additional Information on citygml-tools:

https://github.com/citygml4j/citygml-tools#:~:text=citygml-tools%20is%20a%20command%20line%20utility%20that%20bundles,is%20licensed%20under%20the%20Apache%20License%2C%20Version%202.0.

If the goal is to read and write host files, can you run the process directly on the host? Do you need the intermediate Docker-in-Docker layer (which adds significant complexity, and is kind of unusual)? — David Maze
– David Maze, Commented Sep 2, 2021 at 16:07
@DavidMaze I'm starting the transformation Container from another Container - which is Communicating with the API - and I wanted to avoid to mount the host socket directly to that Container, therefore used the intermediate Dind Container. But If there's no solution doing it this way, mounting the Host socket should be possible too. — BenjaminK
– BenjaminK, Commented Sep 3, 2021 at 7:56

David Maze · Accepted Answer · 2021-09-03 10:47:48Z

The DinD setup you're proposing here is especially complex and I would avoid it. If possible, I'd even avoid launching additional containers here.

In the actual API request, I suspect you have too much punctuation. The Docker CLI and API documentation use square brackets for grouping and indicating optional parts of names; [r]shared means "either shared or rshared". In the API ContainerCreate operation, the "HostConfig": {"Binds: []}" option includes strings with two or three colon-separated parts, /host/path:/container/path:options; the options are a comma-separated list of options, and the :options section is optional. So you may need instead

{
  "HostConfig": {
    "Binds": [
      "/mnt:/data:rw"
    ]
  }
}

with no square brackets at all.

Your DinD setup is mounting a named volume from the host's Docker into the DinD container's /mnt directory, and then your API request is trying to mount that. This won't usually create directly accessible host files and you may also need to change the DinD container to use -v /host:/container bind-mount syntax.

You can use citygml-tools as a (Java) library. If your application is JVM-based, doing this avoids all of the complexity you show here. Bind-mount the host directory with the data into your application container and remove all of the Docker API setup.

If you can't do that, you can also install a JVM and the citygml-tools release into your Docker image itself. Having done this, you can run commands like citygml-tools to-cityjson as an ordinary subprocess, without involving Docker. Again, bind-mount the host directory into the application container.

RUN apt-get update \
 && DEBIAN_FRONTEND=noninteractive \
    apt-get install --no-install-recommends --assume-yes \
      curl \
      openjdk-11-jre-headless \
      unzip
RUN cd /usr/local \
 && curl -LO https://github.com/citygml4j/citygml-tools/releases/download/v1.4.2/citygml-tools-1.4.2.zip \
 && unzip citygml-tools-1.4.2.zip \
 && rm citygml-tools-1.4.2.zip

If you need to keep the Docker API path, a simpler option will be to skip the DinD container and talk directly to the host's Docker socket. This is considered insecure, but precisely because it can escape the Docker container and mount arbitrary files on the host into a new container (even the host's /etc, read-write, with the container running as root). Whichever XML-based tool you're using to make requests would need to know how to make HTTP requests to a Unix socket. (Do NOT enable TCP access to the host Docker socket.)

docker run ... \
  -v /var/run/docker.sock:/var/run/docker.sock \
  your-image
# without a DinD container

You may be hitting some trouble with bind propagation since you're trying to remount a mount point in another context. I don't think this applies (you're mounting the mount point directly and not trying to mount an additional filesystem as a subdirectory of a mount point). If it does, though, this DinD approach won't work at all since you can't control the propagation settings for a named-volume mount.

What is the best Practice for Accessing Host Data [...], and write Data in the Host filesystem [...]?

Run the process directly on the host. This won't require escalated privileges to run, it obeys normal host filesystem permissions, and it will usually be easier to develop and test the application. If you're thinking ahead to clustered environments like Kubernetes, these actually can't access "host data" at all (if your cluster has 100 nodes, which is "the host"?) and you'll need some other storage mechanism.

Hey David, thanks for this Comprehensive Answer! to use this toolbox as a Java library Is not possible for my Use Case. I tried Mounting the Host, but I'm not quite sure how to send a Http Request to the Unix Socket. My tool requires A URL. I tried with var/run/docker.sock, but that did not work. probably a stupid idea, but I had no better.
I connected the socket as you specified. When I Inspect my Contaier, the Bind looks like this: "Type": "bind", "Source": "/run/host-services/docker.proxy.sock", "Destination": "/var/run/docker.sock", "Mode": "", "RW": true, "Propagation": "rprivate" And I still can not connect to the socket. Is there more I need to specitfy/consider? How do I sent a http request to a unix socket?

Collectives™ on Stack Overflow

Mount docker Host path to Container

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related