Doppler: How to Set Environment Variables for a Python Django Application using Apache and mod_wsgi in Docker

Using environment variables to configure Django and other Python applications is awesome, but using them with Apache and mod_wsgi in Docker is a tricky thing to get right.

That's why I created this step-by-step tutorial and sample application to put all the info you need in one place.

Although this tutorial is for Docker and Django, the same steps apply, whether you're using a Virtual Machine or a different Python framework.

Prefer just to read the code? Head to the accompanying repository at https://github.com/DopplerUniversity/django-apache-mod-wsgi

Environment Variables, Apache and mod_wsgi

When hosting a Python WSGI compatible framework like Django in Apache with mod_wsgi, the only environment variables populated in the os.environ dictionary are those that exist in the environment of the script that starts Apache. But instead of having to mess with Apache's service manager settings (e.g. systemd or systemctl), there's a better way.

Most Apache distributions provide a shell script specifically for the purpose of setting environment variables that will be made available to modules such as mod_wsgi.

It's then a matter of knowing the location of this shell script as it can be different depending on the Linux distribution. For example:

  • Debian/Ubuntu: /etc/apache2/envvars
  • CentOS: /etc/sysconfig/httpd

We'll be using the python:3.9-slim-buster Docker image is Debian based.

Appending App Config and Secrets to the Environment Variables File

Essentially, it boils down to fetching the secrets as key/value pairs and writing them to the envvars file in the typical shell environiment variables format:

export FIRST_NAME="The"
export LAST_NAME="Mandalorion"

But where from and how do we fetch the app config and secrets to populate that file?

As I'm the Developer Advocate for Doppler, I'll start with a Doppler CLI example, but the mechanics of "fetch secrets, then append to file" can easily be adapted.

First, you would need to set up your project in Doppler and you can use the following button to get you started if you want to follow along.

Then use the Doppler CLI inside the Docker container to fetch the secrets (requires a DOPPLER_TOKEN environment variable with a Service Token value):

# Transform JSON key:value pairs into export statements using jq
if [ -n "$DOPPLER_TOKEN" ]; then
    echo '[info]: Appending environment variables to /etc/apache/envvars using Doppler CLI'
    doppler secrets download --no-file | jq -r $'. | to_entries[] | "export \(.key)=\'\(.value)\'"' >> /etc/apache2/envvars
fi

Notice I've used single quotes, not double quotes around the values?

That's because it gives you the flexibility of storing secrets with double quotes such as JSON in Doppler which you could use for example, to dynamically set Django's ALLOWED_HOSTS setting dynamically for any environment.

ALLOWED_HOSTS = json.loads(os.environ['ALLOWED_HOSTS'])

You could also use a .env file but I wouldn't recommend it and instead, I'd look into using a secrets manager.

But that aside, here is how you could do it using an .env file:

if [ -f "$PWD/.env" ]; then
    echo '[info]: Appending environment variables to /etc/apache/envvars from .env file'
    cat "$PWD/.env" >> /etc/apache2/envvars
fi

Now that we know how to pass environment variables from Apache to mod_wsgi, let's move onto getting this working in Docker.

Docker Configuration for Apache and mod_wsgi

Let's breakdown the task of configuring a Python Django Application using Apache and mod_wsgi in Docker into three steps:

  1. Custom Start Script
  2. Apache Site Config
  3. Dockerfile

If you only want to see the working code examples, head to the accompanying repository at https://github.com/DopplerUniversity/django-apache-mod-wsgi

As this isn't a Docker or Apache tutorial, I won't be diving too deeply into the Dockerfile or Apache site config file, but if you've got questions, head over to the Doppler community forum and I'll be able to help you there.

1. Custom Start Script

Running your application in Docker is usually a case of setting the CMD, for example:

CMD ["python", "src/app.py"]

But it's trickier here as we first need to append the environment variables to /etc/apache2/envvars before running Apache.

As this requires multiple commands, we'll create a custom script:

#!/bin/bash

# apache-doppler-start

set -e

echo 'ServerName localhost' >> /etc/apache2/apache2.conf # Silence FQDN warning

# Doppler CLI
if [ -n "$DOPPLER_TOKEN" ]; then
    echo '[info]: Appending environment variables to /etc/apache/envvars from Doppler CLI'
    doppler secrets download --no-file | jq -r $'. | to_entries[] | "export \(.key)=\'\(.value)\'"' >> /etc/apache2/envvars
fi

# Mounted .env file
if [ -f "$PWD/.env" ]; then
    echo '[info]: Appending environment variables to /etc/apache/envvars from .env file'
    cat "$PWD/.env" >> /etc/apache2/envvars
fi

# Run Apache
apache2ctl -D FOREGROUND

2. Apache Site Config

Here is an example Apache site config file for a Django application:

# wsgi.conf
<VirtualHost *:80>
    ServerName django-apache-mod-wsgi
    ServerAlias django-apache-mod-wsgi
    ServerAdmin webmaster@doppler

    # Defining `WSGIDaemonProcess` and `WSGIProcessGroup` triggers daemon mode
    WSGIDaemonProcess django-apache-mod-wsgi processes=2 threads=15 display-name=%{GROUP} python-path=/usr/local/lib/python3.9/site-packages:/usr/src/app    
    WSGIProcessGroup django-apache-mod-wsgi
    WSGIScriptAlias / /usr/src/app/doppler/wsgi.py

    <Directory /usr/src/app/doppler/>
        <Files wsgi.py>
            Require all granted
        </Files>
    </Directory>

    # Redirect all logging to stdout for Docker
    LogLevel INFO
    ErrorLog /dev/stdout
    TransferLog /dev/stdout
</VirtualHost>

3. Dockerfile

The Dockerfile is reasonably straightforward, installing the Doppler CLI and Apache dependencies before copying the Django source code, custom script, and Apache site config:

FROM python:3.9-slim-buster

ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1

# Install Doppler CLI and related dependencies
RUN apt-get -qq update && apt-get install -y apt-transport-https ca-certificates curl gnupg jq && \
curl -sLf --retry 3 --tlsv1.2 --proto "=https" 'https://packages.doppler.com/public/cli/gpg.DE2A7741A397C129.key' |  apt-key add - && \
echo "deb https://packages.doppler.com/public/cli/deb/debian any-version main" | tee /etc/apt/sources.list.d/doppler-cli.list && \
apt-get -qq update && apt-get install doppler

# Install Apache and related dependencies
RUN apt-get install --yes apache2 apache2-dev libapache2-mod-wsgi-py3 && \
    apt-get clean && \
    apt-get remove --purge --auto-remove -y && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /usr/src/app

COPY requirements*.txt .
RUN pip install --quiet --no-cache-dir --upgrade pip && \
    pip install --quiet --no-cache-dir -r requirements.txt

# Application source
COPY src/ ./

# Custom CMD script
COPY apache-doppler-start /usr/local/bin/

# Apache site config
COPY wsgi.conf /etc/apache2/sites-enabled/000-default.conf

EXPOSE 80 443

# https://httpd.apache.org/docs/2.4/stopping.html#gracefulstop
STOPSIGNAL SIGWINCH

CMD ["apache-doppler-start"]

With all the pieces in place, we can nowbuild the Docker image (clone the sample repository to follow along):

docker image build -t django-apache-mod-wsgi:latest .

Now we're ready to run the container!

Running the Django Application with Apache and mod_wsgi in Docker

We'll start with a Doppler example, then with an .env file.

With Doppler, you'll first need to set a DOPPLER_TOKEN environment variable to the value of a Service Token. This is what provides read-only access to a specific Doppler config in production environments.

Usually, this would be securely set by your deployment environment (e.g. GitHub Action Secret) but for completeness and simplicity, we'll set it manually below:

export DOPPLER_TOKEN="dp.st.xxxx" # Service token value created from Doppler dashboard

Now run the container:

docker container run \
    -it \
    --init \
    --name doppler-apache-mod-wsgi \
    --rm \
    -p 8080:80 \
    -e DOPPLER_TOKEN="$DOPPLER_TOKEN" \
    django-apache-mod-wsgi

.env File

To run the .env file version, we'll use the sample.env file from the sample repository:

# sample.env
export DJANGO_SETTINGS_MODULE='doppler.settings'
export DEBUG='yes'
export ALLOWED_HOSTS='["*"]'
export SECRET_KEY='bf5e1b31-6ba7-48e2-9175-f2293671e6df'

Then to run the container:

docker container run \
    -it \
    --init \
    --name dotenv-apache-mod-wsgi \
    --rm \
    -v $(pwd)/sample.env:/usr/src/app/.env \
    -p 8080:80 \
    django-apache-mod-wsgi

Summary

Nice work in making it to the end!

Now you know how to configure Python applications hosted with Apache and mod_wsgi running in Docker using environment variables for app configuration and secrets.

Feedback is welcome and you can reach us on Twitter, our Community forum, or send me an email at [email protected].

30

Categories:

PythonDjango

Tags:

PythonDjangoApacheDocker