How to Check if an Exception Is Raised (or Not) With pytest

TL;DR

Time is a precious resource so I won't waste yours. Here's how you can assert an exception is raised and how to check that in pytest.

Solution: Use pytest.raises

import pytest

def test_raises_exception():
    with pytest.raises(ZeroDivisionError):
        1 / 0

And here's how you assert no exception is raised.

Solution: Enclose your code in a try/except block and and if the code raises, you can catch it and print a nice message. pytest is smart enough to make the test fail even if you don't catch it but having a message makes your test cleaner.

def my_division_function(a, b):
    return a / b

def test_code_raises_no_exception():
    """
    Assert your python code raises no exception.    
    """
    try:
        my_division_function(10, 5)
    except ZeroDivisionError as exc:
        assert False, f"'10 / 5' raised an exception {exc}"

And that's it, if you want to know more, please follow along.

Introduction

In this tutorial, you'll learn how to use pytest to:

assert that an exception is raised
assert the exception message
assert the exception type
assert that an exception is not raised

In a nutshell, we'll see how to use pytest.raises for each of those cases with examples.

How to Assert That an Exception Is Raised
How to Assert That NO Exception Is Raised
How to Assert the Exception Message - And Type
Conclusion

How to Assert That an Exception Is Raised

In this section, I’m going to show you how you can assert that your code raises an exception. This is a frequent use case and can sometimes tricky. The wonderful thing is, if you are using pytest you can do that in an idiomatic and cleaner way.

Let’s imagine that we have a function that checks for some keys in a dictionary. If a key is not present, it should raise a KeyError. As you can see, this is very generic and doesn’t tell the users much about the error. We can make it cleaner by raising custom exceptions, with different messages depending on the field.

import pytest


class MissingCoordException(Exception):
    """Exception raised when X or Y is not present in the data."""


class MissingBothCoordException(Exception):
    """Exception raised when both X and Y are not present in the data."""


def sum_x_y(data: dict) -> str:
    return data["x"] + data["y"]

Now, time to test this. How can we do that with pytest?

This code is deliberately wrong, as you can see we’re not raising anything. In fact, we want to see test failing first, almost like TDD. After seeing the test failing, we can fix our implementation and re-run the test.

def test_sum_x_y_missing_both():
    data = {"irrelevant": 1}
    with pytest.raises(MissingBothCoordException):
        sum_x_y(data)

Then we get the following output:

============================ FAILURES ============================
________________ test_sum_x_y_missing_both _________________

    def test_sum_x_y_missing_both():
        data = {"irrelevant": 1}
        with pytest.raises(MissingBothCoordException):
>           sum_x_y(data)

test_example.py:33: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

data = {'irrelevant': 1}

    def sum_x_y(data: dict) -> str:
>       return data["x"] + data["y"]
E       KeyError: 'x'

test_example.py:27: KeyError
==================== short test summary info =====================
FAILED test_example.py::test_sum_x_y_missing_both - KeyEr...
======================= 1 failed in 0.02s ========================

Ok, this makes sense, now it’s time to fix it. We’ll check if the data dict has both x and y, otherwise we raise a MissingBothCoordException.

def sum_x_y(data: dict) -> str:
    if "x" not in data and "y" not in data:
        raise MissingBothCoordException("Both x and y coord missing.")
    return data["x"] + data["y"]

And when we re-run the test, it passes.

test_example.py .                                          [100%]

======================= 1 passed in 0.01s ========================

Great! And that is pretty much it. This is how you check if an exception is raised withpytest. In the next section, we’re going to improve our function and we’ll need another test.

How to Assert the Exception Message - And Type

In this section, we’ll improve our sum_x_y function and also the tests. I’ll show you how you can make your test more robust by checking the exception message.

With that in mind, let’s expand the sum_x_y function.

def sum_x_y(data: dict) -> str:
    if "x" not in data and "y" not in data and "extra" not in data:
        raise MissingBothCoordException("Both X and Y coord missing.")
    if "x" not in data:
        raise MissingCoordException("The Y coordinate is not present in the data.")
    if "y" not in data:
        raise MissingCoordException("The Y coordinate is not present in the data.")
    return data["x"] + data["y"]

The new test goes like this:

def test_sum_x_y_has_x_missing_coord():
    data = {"extra": 1, "y": 2}
    with pytest.raises(MissingCoordException):
        sum_x_y(data)

And it passes!

$ poetry run pytest -k test_sum_x_y_has_x_missing_coord
====================== test session starts =======================
collected 2 items / 1 deselected / 1 selected                    

test_example.py .                                          [100%]

================ 1 passed, 1 deselected in 0.01s =================

However, it’s a bit fragile... In case you haven’t noticed it, when "x" is missing, the exception message is: "The Y coordinate is not present in the data.". This is a bug, and one way to detect it is by asserting we return the right message. Thankfully, pytest makes it easier to do.

If we refactor the test to take into account the message, we get the following output:

def test_sum_x_y_has_x_missing_coord():
    data = {"extra": 1, "y": 2}
    with pytest.raises(MissingCoordException) as exc:
        sum_x_y(data)
    assert "The X coordinate is not present in the data." in str(exc.value)

============================ FAILURES ============================
_____________ test_sum_x_y_has_x_missing_coord _____________

def test_sum_x_y_has_x_missing_coord():
        data = {"extra": 1, "y": 2}
        with pytest.raises(MissingCoordException) as exc:
            sum_x_y(data)
>       assert "The X coordinate is not present in the data." in str(exc.value)
E       AssertionError: assert 'The X coordinate is not present in the data.' in 'The Y coordinate is not present in the data.'
E        +  where 'The Y coordinate is not present in the data.' = str(MissingCoordException('The Y coordinate is not present in the data.'))
E        +    where MissingCoordException('The Y coordinate is not present in the data.') = <ExceptionInfo MissingCoordException('The Y coordinate is not present in the data.') tblen=2>.value

test_example.py:32: AssertionError
==================== short test summary info =====================
FAILED test_example.py::test_sum_x_y_has_x_missing_coord
======================= 1 failed in 0.02s ========================

That's exactly what we want. Let's fix the code and re-run the test.

def sum_x_y(data: dict) -> str:
    if "x" not in data and "y" not in data and "extra" not in data:
        raise MissingBothCoordException("Both X and Y coord missing.")
    if "x" not in data:
        raise MissingCoordException("The X coordinate is not present in the data.")
    if "y" not in data:
        raise MissingCoordException("The Y coordinate is not present in the data.")
    return data["x"] + data["y"]

And the result...

$ poetry run pytest test_example.py::test_sum_x_y_has_x_missing_coord
====================== test session starts =======================
platform linux -- Python 3.8.5, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/miguel/projects/tutorials/pytest-raises
collected 1 item                                                 

test_example.py .                                          [100%]

======================= 1 passed in 0.01s ========================

This is possible because pytest.raises returns an ExceptionInfo object that contains fields such as type, value, traceback and many others. If we wanted to assert the type, we could do something along these lines...

def test_sum_x_y_has_x_missing_coord():
    data = {"extra": 1, "y": 2}
    with pytest.raises(MissingCoordException) as exc:
        sum_x_y(data)
    assert "The X coordinate is not present in the data." in str(exc.value)
    assert exc.type == MissingCoordException

However, we are already asserting that by using pytest.raises so I think asserting the type like this a bit redundant. When is this useful then? It's useful if we are asserting a more generic exception in pytest.raises and we want to check the exact exception raised. For instance:

def test_sum_x_y_has_x_missing_coord():
    data = {"extra": 1, "y": 2}
    with pytest.raises(Exception) as exc:
        sum_x_y(data)
    assert "The X coordinate is not present in the data." in str(exc.value)
    assert exc.type == MissingCoordException

One more way to assert the message is by setting the match argument with the pattern you want to be asserted. The following example was taken from the official pytest docs.

>>> with raises(ValueError, match='must be 0 or None'):
...     raise ValueError("value must be 0 or None")

>>> with raises(ValueError, match=r'must be \d+$'):
...     raise ValueError("value must be 42")

As you can see, we can verify if the expected exception is raised but also if the message matches the regex pattern.

How to Assert That NO Exception Is Raised

The last section in this tutorial is about yet another common use case: how to assert that no exception is thrown. One way we can do that is by using a try / except. If it raises an exception, we catch it and assert False.

def test_sum_x_y_works():
    data = {"extra": 1, "y": 2, "x": 1}

    try:
        sum_x_y(data)
    except Exception as exc:
        assert False, f"'sum_x_y' raised an exception {exc}"

When we run this test, it passes.

$ poetry run pytest test_example.py::test_sum_x_y_works
====================== test session starts =======================
collected 1 item                                                 

test_example.py .                                          [100%]

======================= 1 passed in 0.00s ========================

Now, let's create a deliberate bug so we can see the test failing. We'll change our function to raise an ValueError before returning the result.

def sum_x_y(data: dict) -> str:
    if "x" not in data and "y" not in data and "extra" not in data:
        raise MissingBothCoordException("'extra field and x / y coord missing.")
    if "x" not in data:
        raise MissingCoordException("The X coordinate is not present in the data.")
    if "y" not in data:
        raise MissingCoordException("The Y coordinate is not present in the data.")
    raise ValueError("Oh no, this shouldn't have happened.")
    return data["x"] + data["y"]

And then we re-run the test...

def test_sum_x_y_works():
        data = {"extra": 1, "y": 2, "x": 1}

        try:
            sum_x_y(data)
        except Exception as exc:
>           assert False, f"'sum_x_y' raised an exception {exc}"
E           AssertionError: 'sum_x_y' raised an exception Oh no, this shouldn't have happened.
E           assert False

test_example.py:52: AssertionError
==================== short test summary info =====================
FAILED test_example.py::test_sum_x_y_works - AssertionErr...
======================= 1 failed in 0.02s ========================

It works! Our code raised the ValueError and the test failed!

Conclusion

That’s it for today, folks! I hope you’ve learned something new and useful. Knowing how to test exceptions is an important skill to have. The way pytest does that is, IMHO, cleaner than unittest and much less verbose. In this article, I showed how you can not only assert that your code raises the expected exception, but also assert when they’re not supposed to be raised. Finally, we saw how to check if the exception message is what you expect, which makes test cases more reliable.