Setup postgres in Github Actions for Django

Question

I'm currently working on a website right now on Django. On my computer, I am running it on Docker with a postgres database. Here's the docker-compose file I have:

version: '3'

services:
    db:
        image: postgres
        environment:
            - POSTGRES_DB=postgres
            - POSTGRES_USER=postgres
            - POSTGRES_PASSWORD=postgres


    web:
        build: .
        volumes:
            - .:/usr/src/app
        ports:
            - "8000:8000"

And here's the relevant part in settings.py

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'postgres',
        'USER': 'postgres',
        'PASSWORD': 'postgres',
        'HOST': 'db',
        'PORT': 5432,
    }
}

When I run my tests in the docker container with this setup, it works find and the tests run. However, in github actions, it doesn't work. Here's my workflow file:

name: Django CI

on: push


jobs:
  build:

    runs-on: ubuntu-latest

    strategy:
      max-parallel: 4
      matrix:
        python-version: [3.7, 3.8]

    services:
      db:
        image: postgres
        env:
          POSTGRES_DB: postgres
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
        ports:
          - 5432:5432

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v1
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install Dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Run Tests
      run: |
        python manage.py test

When this runs in github actions, I get the following error:

django.db.utils.OperationalError: could not translate host name "db" to address: Temporary failure in name resolution

Could someone please help me with this, and please let me know if you need anymore code.

Chokefull

what happens if you use localhost? – Eous

On my computer I run it on 127.0.0.1:8000, the default for Django and it works fine. – Chokefull

I meant in github actions – Eous

Oh, thanks for the suggestion. For some reason, it says ModuleNotFoundError: No module named 'localhost:8000'. I think that maybe you can't specify localhost in tests? – Chokefull

Answer 1

The reason this is not working is that the settings HOST is set to db.

When you're working with a Github Action that only has services, this is called "running all steps on the virtual machine" or VM.

You know this is the case because your steps are at the same indentation level as services.

In this situation, you do not specify the service's container name (in this case db) as the host. Instead, services running in services as shown in the question are actually running on localhost which is the same place the steps are taking place.

When the run tests step is reached, we can see that django is in fact attempting to start up, because the django.db.utils.OperationalError... error is a Django runtime error. It is showing Django's attempt to reach out and talk to the database.

This means that the settings configuration is wrong. To specifically answer this question, simply update the settings to:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'postgres',
        'USER': 'postgres',
        'PASSWORD': 'postgres',
        'HOST': 'localhost',
        'PORT': 5432,
    }
}

Note a very similar question and resolving answer is shown here.

---

If this doesn't fix it

If this is still not working for you, most likely the settings.py that the Github Actions is running is not the same one you are editing.

As I mention here, ensure first that your setting is not overridden and that your Github Actions job is pulling the correct branch that you are editing the settings.py.

This can be especially hard because of the delay of waiting for Github Actions to build causes something of a disconnect between what you're doing and the result.

You may think you're working on relevant code / pushing to the right place / setting the correct environment variables when you are not. And you do not detect it possibly because it is easy to get distracted waiting to see if your fix worked. :)

One more note on this, Github has an example simple service using postgres. Look at it carefully. The second example shows the one asked about in this question, where all steps are run on the virtual machine.

Unexpressed

Thanks! I think I've found an official explanation, docs: docs.github.com/en/actions/using-containerized-services/… – Kuibyshev

Answer 2

As @Rob pointed out the right point, the configuration between settings.py and PostgreSQL running in the container is wrong. In short, PostgreSQL db in a container does not access your secret key so you need to give them explicitly.

settings.py

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': env('POSTGRES_DB'),
        'USER': env('POSTGRES_USER'),
        'PASSWORD': env('POSTGRES_PASSWORD'),
        'HOST': env('POSTGRES_HOST'),
        'PORT': 5432
    }
}

I use django-environ to store my environment variables. os.getenv also works here. Just change os.getenv with env.

In your workflow file with the extension of .yml, services section should seem like this:

services:
  postgres:
    image: postgres:12.5
    env:
      POSTGRES_DB: postgres
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
    ports:
      - 5432:5432
      # needed because the postgres container does not provide a healthcheck
    options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

In the same file, it is OK to use all environmental variables in the secure way. For example, this shows my Django Testing workflow:

- name: Django Testing
  env:
    DEBUG: ${{ secrets.DEBUG }}
    SECRET_KEY: ${{ secrets.SECRET_KEY }}
    POSTGRES_HOST: ${{ secrets.POSTGRES_HOST }}
    POSTGRES_DB: ${{ secrets.POSTGRES_DB }}
    POSTGRES_USER: ${{ secrets.POSTGRES_USER }}
    POSTGRES_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
  run: |
    python manage.py test

Lastly, in the Github repository secrets, define your variables as :

POSTGRES_HOST --> localhost
POSTGRES_DB --> postgres
POSTGRES_USER --> postgres
POSTGRES_PASSWORD --> postgres
DEBUG --> True
SECRET_KEY --> <Your Django Secret Key>

Finally, you will have:

Understudy