Docker - output data to a file - Python

Question

I am starting out with docker and trying to run a python code that generates a csv file as output.

import pandas as pd  
  
# string values in the list   
lst = ['Java', 'Python', 'C', 'C++',  
         'JavaScript', 'Swift', 'Go']  
  
# Calling DataFrame constructor on list  
dframe = pd.DataFrame(lst)  
dframe.to_csv('test.csv')

This is my docker configuration:

FROM python:3.8

ADD main.py .
VOLUME /out
RUN pip install pandas

CMD ["python", "./main.py"]

This is how I call docker.

docker run  --volume=/Users/myuser/Documents/DashyDash/Docker/out:/out python-test

However I am unable to find the output file. Can someone tell me what I am doing wrong and help me fix it.

Thank you!

The to_csv is writing to current dir and not to out. One suggestion is also to not use volume but rather create and write to an output folder and mount that. — coldy
– coldy, Commented Feb 6, 2022 at 20:39
Thanks @coldy That was my next question whether the way I did is the best way. How do I do what you are suggesting? — hsnsd
– hsnsd, Commented Feb 6, 2022 at 20:50
A Docker container has an isolated filesystem; it by design is difficult to read and write host files. MacOS and Linux systems generally come with Python preinstalled. Can you use a Python virtual environment on the host system to run this script, without Docker? That setup would be able to read and write files normally. — David Maze
– David Maze, Commented Feb 7, 2022 at 0:27

coldy · Accepted Answer · 2022-02-06 21:17:25Z

1

The location of the main.py writes the csv in the current directory which has no mapping to know which directory to the host. Therefore, you will have to change that so that the same is reflected in the host machine. I assume my work directory within the Dockerfile is /app and I also create a new directory output which is then mapped as a bind mount volume while running the container. I will detail that next.

Dockerfile: Same things with few improvements

FROM python:3.8
WORKDIR /app # This makes sure that a particular work dir is set
COPY main.py ./ # Copying all required files, could also be directories
RUN mkdir -p /app/output # Creating a new directory called output
RUN pip install pandas
CMD ["python3", "main.py"] # Just small improvement

Now, I map the same output to the main.py: dframe.to_csv('/app/output/test.csv')

After this, I build the image:

docker build -t python-test .

Run the image:

docker run -v $HOME/test/output:/app/output python-test

Note: The entire files are on $HOME/test. Also, main.py can map the output directory automatically as well using python's os package.

Then, the output in the output directory is:

,0
0,Java
1,Python
2,C
3,C++
4,JavaScript
5,Swift
6,Go

answered Feb 6, 2022 at 21:17

coldy

2,2352 gold badges19 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

hsnsd Over a year ago

Thanks alot for this @coldy

Collectives™ on Stack Overflow

Docker - output data to a file - Python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related