Understand Django: Command Your App

In the last Understand Django article, we dug into file management. We saw how Django handles user uploaded files and how to deal with them safely.

With this article, you'll learn about commands. Commands are the way to execute scripts that interact with your Django app. We'll see built-in commands and how to build your own.

Why Commands?

Django makes it possible to run code from a terminal with ./manage.py, but why is this helpful or needed? Consider this script:

# product_stat.py

from application.models import Product

print(Product.objects.count())

which you could try running with:

$ python product_stat.py

The problem with this script is that Django is not ready to run yet. If you tried to run this kind of code, you would get an ImproperlyConfigured exception. There are a couple of modifications that you could make to get the script to run.

• Call django.setup().
• Specify the DJANGO_SETTINGS_MODULE.

# product_stat.py
import django

django.setup()

from application.models import Product

print(Product.objects.count())

Note that django.setup() must be before your Django related imports (like the Product model in this example). Now the script can run if you supply where the settings are located too.

$ DJANGO_SETTING_MODULE=project.settings python product_stat.py

This arrangement is less than ideal, but why else might we want a way to run commands through Django?

Try running ./manage.py -h.

What you'll likely see is more commands than what Django provides alone. This is where we begin to see more value from the command system. Because Django provides a standard way to run scripts, other Django applications can bundle useful commands and make them easily accessible to you.

Now that you've had a chance to see why commands exist to run scripts for Django apps, let's back up and see what commands are.

We Hereby Command

Django gives us a tool to run commands before we've even started our project. That tool is the django-admin script. We saw it all the way back in the first article where I provided a short set of setup instructions to get you started if you've never used Django before.

After you've started a project, your code will have a manage.py file, and the commands you've seen in most articles are in the form of:

$ ./manage.py some_command

What's the difference between django-admin and manage.py? In truth, not much!

django-admin comes from Django's Python packaging. In Python packages, package developers can create scripts by defining an entry point in the packaging configuration. In Django, this configuration looks like:

[options.entry_points]
console_scripts =
    django-admin = django.core.management:execute_from_command_line

Meanwhile, the entire manage.py of one of my projects is:

#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys


def main():
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)


if __name__ == '__main__':
    main()

If you look closely, you can see that the different scripts are both ways to invoke the execute_from_command_line function to run a command. The primary difference is that the latter script will attempt to set the DJANGO_SETTINGS_MODULE environment variable automatically. Since Django needs to have DJANGO_SETTINGS_MODULE defined for most commands (note: startproject does not require that variable), manage.py is a more convenient way to run commands.

execute_from_command_line is able to present what commands are available to a project, whether a command comes from Django itself, an installed app, or is a custom command that you created yourself. How are the commands discovered? The command system does discovery by following some packaging conventions.

Let's say your project has an app named application. Django can find the command if you have the following packaging structure.

application
├── __init__.py
├── management
│   ├── __init__.py
│   └── commands
│       ├── __init__.py
│       └── custom_command.py
├── models.py
└── views.py
... Other typical Django app files

With this structure, you could run:

$ ./manage.py custom_command

Notes:

• Django will create a command for a module found in <app>/management/commands/<command name>.py.
• Don't forget the __init__.py files! Django can only discover the commands if management and commands are proper Python package directories.
• The example uses custom_command, but you can name your command with whatever valid Python module name that you want.

Unfortunately, we can't slap some Python code into custom_command.py and assume that Django will know how to run it. Within the custom_command.py module, Django needs to find a Command class that subclasses a BaseCommand class that is provided by the framework. Django requires this structure to give command authors a consistent way to access features of the command system.

With the Command class, you can add a help class attribute. Adding help can gives users a description of what your command does when running ./manage.py custom_command -h.

The Command class will also help you with handling arguments. If your command needs to work with user input, you'll need to parse that data. Thankfully, the class integrates with Python's built-in argparse module. By including an add_arguments method, a command can parse the data and pass the results to the command's handler method in a structured way. If you've had to write Python scripts before, then you may understand how much time this kind of parsing can save you (and for those who haven't, the answer is "a lot of time!").

Other smaller features exist within the Command class too. Perhaps you only want your command to run if your project has satisified certain pre-conditions. Commands can use the requires_migration_checks or requires_system_checks to ensure that the system is in the correct state before running.

I hope it's clear that the goal of the Command class is to help you with common actions that many commands will need to use. There is a small API to learn, but the system is a boon to making scripts quickly.

Command By Example

Let's consider a powerful use case to see a command in action. When you initially start a Django app, all of your app's interaction will probably be through web pages. After all, you were trying to use Django to make a web app, right? What do you when you need to do something that doesn't involve a browser?

This kind of work for your app is often considered background work. Background work is a pretty deep topic and will often involve special background task software like Celery. When your app is an early stage, Celery or similar software can be overkill and far more than you need.

A simpler alternative for some background tasks could be a command paired with a scheduling tool like cron.

On one of my projects, I offer free trials for accounts. After 60 days, the free trial ends and users either need to pay for the service or discontinue using it. By using a command and pairing it with the Heroku Scheduler, I can move accounts from their trial status to expired with a daily check.

The following code is very close to what my expire_trials command looks like in my app. I've simplified things a bit, so that you can ignore the details that are specific to my service.

# application/management/commands/expire_trials.py

import datetime

from django.core.management.base import BaseCommand
from django.utils import timezone

from application.models import Account


class Command(BaseCommand):
    help = "Expire any accounts that are TRIALING beyond the trial days limit"

    def handle(self, *args, **options):
        self.stdout.write("Search for old trial accounts...")
        # Give an extra day to be gracious and avoid customer complaints.
        cutoff_days = 61
        trial_cutoff = timezone.now() - datetime.timedelta(days=cutoff_days)
        expired_trials = Account.objects.filter(
            status=Account.TRIALING, created__lt=trial_cutoff
        )
        count = expired_trials.update(status=Account.TRIAL_EXPIRED)
        self.stdout.write(f"Expired {count} trial(s)")

I configured the scheduler to run python manage.py expire_trials every day in the early morning. The command checks the current time and looks for Account records in the trialing state that were created before the cutoff time. From that query set, the affected accounts are set to the expired account state.

How can you test this command? There are a couple of approaches you can take when testing a command.

If you need to simulate calling the command with command line arguments, then you can use call_command from django.core.management. Since the example command doesn't require arguments, I didn't take that approach.

Generally, my preference is to create a command object and invoke the handle method directly. In my example above, you can see that the command uses self.stdout instead of calling print. Django does this so that you could check your output if desired.

Here is a test for this command:

# application/tests/test_commands.py

from io import StringIO

from application.models import Account
from application.tests.factories import AccountFactory

def test_expires_trials():
    """Old trials are marked as expired."""
    stdout = StringIO()
    account = AccountFactory(
        status=Account.TRIALING,
        created=timezone.now() - datetime.timedelta(days=65),
    )
    command = Command(stdout=stdout)

    command.handle()

    account.refresh_from_db()
    assert account.status == Account.TRIAL_EXPIRED
    assert "Expired 1 trial(s)" in stdout.getvalue()

In this test, I constructed a command instance and checked the account state after the command invocation. Also, observe that the StringIO instance is injected into the Command constructor. By building the command this way, checking the output becomes a very achievable task via the getvalue method.

Overall, this scheme of making a command and running it on a schedule avoids all the work of setting up a background worker process. I've been extremely satisfied with how this technique has worked for me, and I think it's a great pattern when your app doesn't have to do a lot of complex background processing.

Useful Commands

Django is full of useful commands that you can use for all kinds of purposes. Thus far in this series, we've discussed a bunch of them, including:

• check - Checks that your project is in good shape.
• collectstatic - Collects static files into a single directory.
• createsuperuser - Creates a super user record.
• makemigrations - Makes new migration files based on model changes.
• migrate - Runs any unapplied migrations to your database.
• runserver - Runs a development web server to check your app.
• shell - Starts a Django shell that allows you to use Django code on the command line.
• startapp - Makes a new Django app from a template.
• startproject - Makes a new Django project from a template.
• test - Executes tests that checks the validity of your app.

Here is a sampling of other commands that I find useful when working with Django projects.

dbshell

The dbshell command starts a different kind of shell. The shell is a database program that will connect to the same database that your Django app uses. This shell will vary based on your choice of database.

For instance, when using PostgreSQL, ./manage.py dbshell will start psql. From this shell, you can execute SQL statements directly to inspect the state of your database. I don't reach for this command often, but I find it very useful to connect to my database without having to remember database credentials.

showmigrations

The showmigrations command has a simple job. The command shows all the migrations for each Django app in your project. Next to each migration is an indicator of whether the migration is applied to your database.

Here is an example of the users app from one of my Django projects:

$ ./manage.py showmigrations users
users
 [X] 0001_initial
 [X] 0002_first_name_to_150_max
 [ ] 0003_profile

In my real project, I've applied all the migrations, but for this example, I'm showing the third migration as it would appear if the migration wasn't applied yet.

showmigrations is a good way to show the state of your database from Django's point of view.

sqlmigrate

The sqlmigrate command is very handy. The command will show you what SQL statements Django would run for an individual migration file.

Let's see an example. In Django 3.1, the team changed the AbstractUser model so that the first_name field could have a maximum length of 150 characters. Anyone using the AbstractUser model (which includes me) had to generate a migration to apply that change.

From my showmigrations output above, you can see that the second migration of my users app applied this particular framework change.

To see the Postgres SQL statements that made the change, I can run:

$ ./manage.py sqlmigrate users 0002
BEGIN;
--
-- Alter field first_name on user
--
ALTER TABLE "users_user" ALTER COLUMN "first_name" TYPE varchar(150);
COMMIT;

From this, we can tell that Postgres executed an ALTER COLUMN DDL statement to modify the length of the first_name field.

squashmigrations

Django migrations are a stack of separate database changes that produce a final desired schema state in your database. Over time, your Django apps will accumulate migration files, but those files have a shelf life. The squashmigrations command is designed to let you tidy up an app's set of migration files.

By running squashmigrations, you can condense an app's migrations into a significantly smaller number. The reduced migrations can accurately represent your database schema, and make it easier to reason about what changes happened in the app's history. As a side benefit, migration squashing can make Django's migration handling faster because Django gets to process fewer files.

Even More Useful Commands

The commands above come with the standard Django install. There's even more cool stuff out there to help with your project development!

A package that I often reach for with my Django projects is the django-extensions package. This package is fully of goodies, including some great optional commands that you can use!

A couple of my favorites include:

shell_plus

How often do you fire up a Django shell, import a model, then do some ORM queries to see the current state of the database? This is something I do quite often.

The shell_plus command is like the regular shell, but the command will import all your models automatically. For the five extra characters of _plus, you can save your fingers a lot of typing to import your models and get directly to whatever you needed the shell for.

The command will also import some commonly used Django functions and features like reverse, settings, timezone, and more.

graph_models

When I'm live streaming my side projects on my Twitch channel, I will often want to show the model relationships of my Django project. With the graph_models command, I can create an image of all my models and how those models relate to each other (using UML syntax). This is a great way to:

• Remind myself of the data modeling choices in my apps.
• Orient others to what I'm doing with my project.

This particular command requires some extra setup to install the right tools to create images, but the setup is manageable and the results are worth it.

Aside from shell_plus and graph_models, there are 20 other commands that you can use that may be very useful to you. You should definitely check out django-extensions.

Summary

In this article, you saw Django commands. We covered:

• Why commands exist in the Django framework
• How commands work
• How to create your own custom command and how to test it
• Useful commands from the core framework and the django-extensions package

In the next article, I'm going to describe internationalization. Internationalization is the process of making your app work for multiple spoken languages. You'll learn about:

• Enabling internationalization for your app
• Tools to help manage an internationalized app
• Extra considerations like handling time formatting

If you'd like to follow along with the series, please feel free to sign up for my newsletter where I announce all of my new content. If you have other questions, you can reach me online on Twitter where I am @mblayman.