How to access my local system Postgres from Docker

Question

When I use Docker on my own machine, I want to access my system's Postgres. This is prior to deploying my images to AWS, where I want to use RDS.

I have followed the steps on Docker's Quickstart: Docker Compose and Rails page with the exception that, because I was working with an existing application, I didn't create the initial minimal Gemfile that just loads Rails.

Outside of the Docker container, Postgres is installed and working correctly:

$ which postgres
/Applications/Postgres.app/Contents/Versions/latest/bin/postgres
$ which pg_restore
/Applications/Postgres.app/Contents/Versions/latest/bin/pg_restore
$ which pg_dump
/Applications/Postgres.app/Contents/Versions/latest/bin/pg_dump

The app has been working fine without Docker for years. To test the app outside of the Docker container once more, I have comment out one line of Docker's recommended docker-compose.yml file as follows:

development: &default
  adapter: postgresql
  encoding: unicode
  database: postgres
  pool: 5
  username: postgres
  password:
  # host: db

test:
  <<: *default
  database: myapp_test

When I do that, I can run rake db:test:clone (for example) outside of the container no worries:

But when I uncomment that line and connect to a TTY in the container, I'm in all sorts of pain:

/myapp# rake db:test:clone                                          
pg_dump -s -x -O -f /myapp/db/structure.sql postgres
Please check the output above for any errors and make sure that `pg_dump` is installed in your PATH and has proper permissions.
/myapp# which postgres
/myapp# which pg_restore
/myapp# which pg_dump
/myapp#

What can I do?

The Dockerfile, based on Docker's instructions with a few extra binaries required by my existing app, is:

FROM ruby:2.3.1-slim

RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
RUN apt-get update
RUN apt-get install -qq -y --no-install-recommends --fix-missing apt-utils
RUN apt-get install -qq -y --no-install-recommends --fix-missing build-essential
RUN apt-get install -qq -y --no-install-recommends --fix-missing git
RUN apt-get install -qq -y --no-install-recommends --fix-missing xvfb
RUN apt-get install -qq -y --no-install-recommends --fix-missing qt5-default
RUN apt-get install -qq -y --no-install-recommends --fix-missing libqt5webkit5-dev
RUN apt-get install -qq -y --no-install-recommends --fix-missing gstreamer1.0-plugins-base
RUN apt-get install -qq -y --no-install-recommends --fix-missing gstreamer1.0-tools
RUN apt-get install -qq -y --no-install-recommends --fix-missing gstreamer1.0-x
RUN apt-get install -qq -y --no-install-recommends --fix-missing nodejs
RUN apt-get install -qq -y --no-install-recommends --fix-missing libpq-dev

RUN mkdir /myapp
WORKDIR /myapp

ADD Gemfile /myapp/Gemfile
ADD Gemfile.lock /myapp/Gemfile.lock
RUN bundle install

ADD . /myapp

Related: I can see redis-cli outside the container but not inside it. This will be an issue too once I solve the Postgres situation.

Update

When I go in like this...

docker-compose run web rake db:create
docker-compose run web rake db:test:clone

...it seems I can access Postgres, but not pg_dump and other utilities required for rake db:test:clone, rake db:test:load or similar.

Update II

I added the following to the Dockerfile:

RUN apt-get install -qq -y --no-install-recommends --fix-missing postgresql-client

Now, instead of giving me a "can't be found" error, it says:

pg_dump: server version: 9.5.4; pg_dump version: 9.4.9
pg_dump: aborting because of server version mismatch

This is an issue you can find discussed elsewhere, but not in the context of Docker, which is crucial here.

Would it help if I downgraded my Mac's Postgres to the version fetched by apt-get? Upgrading the version installed by apt-get appears not to be an option:

Step 15 : RUN apt-get install -qq -y --no-install-recommends --fix-missing postgresql-client-9.5
 ---> Running in 614b88701710
E: Unable to locate package postgresql-client-9.5
E: Couldn't find any package by regex 'postgresql-client-9.5'

squarism · Accepted Answer · 2016-09-07 19:41:17Z

2

This isn't "yer doin it wrong", this is tricky. I ran through the docker intro you linked and I couldn't get it to work.

Here's what I did for our app. I built our app off of a base image named after the app. The base image is named after the app. So using your example it'd be called myapp_base. I don't see your docker-compose.yml file here so I'll make some assumptions.

web: image: your_company/myapp_base links: - db volumes: - .:/myapp # ports: # - "3002:3002" # env_file: # - .myapp.env

Port and env_file are not needed but they might come in handy later. Myapp_base is built with a weird workflow that pulls in deps but basically throws in postgres-client if the app is declared to need postgres. It's in chef but it's basically myapp runlist needs postgresql. We're actually moving away from this and trying to put the setup stuff in scripts and the Dockerfile itself.

So ... basically just add the apt-get install postgres-client to your Dockerfile, tag it as myapp_base and use that in your docker-compose.yml file. Then run the docker-compose run web rake db stuff and psql will be found.

edited Sep 7, 2016 at 19:41

answered Sep 6, 2016 at 23:51

squarism

3,3074 gold badges29 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

steven_noble Over a year ago

Thanks. This looks in principle like it should work. (But then, so did the Docker docs.) But I ended up deploying to ElasticBeanstalk without Docker in the end.

Collectives™ on Stack Overflow

How to access my local system Postgres from Docker

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related