Pycharm use Docker Container Python as Remote Interpreter

Question

I am trying to use the python in a docker container on a remote machine as the interpreter in Pycharm. Since that is a mouthful, here is a diagram:

There is a Jupyter Notebook running in the container, which I am able to connect to through my local browser (although this is just for testing the connection). The command I am using to launch the Docker container is

docker run --runtime=nvidia -it --rm --shm-size=2g -v /home/timo/storage:/storage -v /etc/passwd:/etc/passwd -v /etc/group:/etc/group --ulimit memlock=-1 -p 8888:8888 -p 7722:22 --ipc=host latest:latest

I can forward the port 8888 which the Jupyter notebook is running on with ssh -L 8888:0.0.0.0:8888 BBB.BBB.BBB.BBB and thus use it on the local machine. But I don't much like using Jupyter for developing and would like to use the Python interpreter in the Docker Container in Pycharm.

When I select "Add Python Interpreter" in Pycharm, I get the following options:

The documentation for Pycharm suggests using the "Add Python Interpreter/Docker" tool which looks like this:

However the documentation doesn't say how to set up the Docker container and the connections if the Docker is on a remote machine.

So my questions are: should I use a Unix or a TCP socket to connect to my remote docker? Or should I somehow forward all the relevant ports from the container and use the "SSH Interpreter" option? And if so, how do I set this all up? Am I setting up my Docker Container properly in the first place?

I think I have trawled through every forum and online resource, over the last two days, but have not come any closer to getting this to work. I have also tried to get this to work in Spyder, but to no avail either. So any advice is very appreciated!

Many thanks!

Perhaps the workaround in youtrack.jetbrains.com/issue/PY-33489 may help, PyCharm doesn't support this usecase natively out of the box so some extra mouse clicks are required. Meanwhile, nvidia is not supported as well youtrack.jetbrains.com/issue/PY-26429 — Pavel Karateev
– Pavel Karateev, Commented Jul 23, 2019 at 15:57
@Pavel_Karateev Thanks for your reply. It is helpful to at least know that this is not trivial and I haven't just been making some stupid mistake. I'll give this a try. — Mr Squid
– Mr Squid, Commented Jul 23, 2019 at 23:11
@Pavel_Karateev Tried this on my private local machine and it works. However it's not a solution for me, since it requires sudo access which I don't have on the servers I ultimately wish to use. Plus it doesn't seem very secure...thanks anyways. — Mr Squid
– Mr Squid, Commented Jul 24, 2019 at 0:46
I have just embarked on this as I am trying to do the same. I have been using Remote Container Development with VS Code. Have you guys managed to find a more user-friendly solution that doesn't require sudo on the remote? — jayBana
– jayBana, Commented Jul 15, 2020 at 19:23

jz314 · Accepted Answer · 2021-06-09 04:39:27Z

Thank you for depicting the dilemma so poignantly and clearly in your cartoon :-). My colleague and I were trying to do something similar and what ultimately worked beautifully was creating an SSH config directly to the Docker container jumping from the remote machine, and then setting it as a remote SSH interpreter so that pycharm doesn't even realize it's a Docker container. It also works well for vscode.

set up ssh service in docker container (subset of steps in https://dev.to/s1ntaxe770r/how-to-setup-ssh-within-a-docker-container-i5i, port22 stuff wasn't needed)
- docker exec -it <container> bash: create admin interactive prompt for docker
- apt-get install openssh-server
- service ssh start
- confirm with service ssh status -> * sshd is running
determine IP and test SSHing from remote machine into container (adapted from https://phoenixnap.com/kb/how-to-ssh-into-docker-container, steps 2 and 3)
- from normal command prompt on remote machine (not within container): docker inspect -f "{{ .NetworkSettings.IPAddress }}" <container> to get container IP
- test: ping -c 3 <container_ip>
- ssh: ssh <container_ip>; should drop you into the container as your user; however, requires container to be configured properly (docker run cmd has -v /etc/passwd:/etc/passwd:ro \ etc.). It may ask for a password. note: if you do this for a different container later that is assigned the same IP, you will get a warning and may need to delete the previous key from known_hosts; just follow the instructions in the warning.
test SSH from local machine
- if you don't have it set up already, set up passwordless ssh key-based authentication to the remote machine with the docker container
- make SSH command that uses your remote machine as a jump server to the container: ssh -J <remote_machine> <container_ip>, as described in https://wiki.gentoo.org/wiki/SSH_jump_host; if successful you should drop into the container just as you did from the remote machine
- save this setup in your ~/.ssh/config; follow the ProxyJump Example from https://wiki.gentoo.org/wiki/SSH_jump_host
- test config with ssh <container_host_name_defined_in_ssh_config>; should also drop you into interactive container
configure pycharm (or vscode or any IDE that accepts remote SSH interpreter)
- Preferences -> Project -> Python Interpreter -> Add -> SSH Interpreter -> New server configuration
- host: <container_host_name_defined_in_ssh_config>
- port: 22
- username: <username_on_remote_server>
- select interpreter, can navigate using the folder icon, which will walk you through paths within the docker, or you can enter the result of which python from the container
- follow pycharm prompts

Thanks for taking the time to post an answer to this so long after the question was asked.

Collectives™ on Stack Overflow

Pycharm use Docker Container Python as Remote Interpreter

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related