Automate Clean Code and Linting in Python

We will be looking at some packages to format, lint, test our code and later create a pre-commit hook to automate the process.
Before we talking about the packages we will use, let's look at the python files we will work with
We have a function called helpers.py
def add(x, y):
return x + y
def subtract(x, y):
return x - y
def multiply(x, y):
return x * y
def divide(x, y):
return x / y
view raw helpers.py hosted with ❤ by GitHub
It has simple arithmetic functions. All the functions accept two parameters and perform an arithmetic operation on them.
We have another file called tester_helpers.py
from .helpers import add, divide, multiply, subtract
def test_add():
assert add(1, 2) == 3
def test_subtract():
assert subtract(2, 1) == 1
def test_multiply():
assert multiply(2, 2) == 4
def test_divide():
assert divide(4, 2) == 2
This file simply tests the functions we defined earlier. It uses assert to do a simple equality check.
Now let's look at the packages we will be using.
Testing 🧪
Pytest 7.7k+ ⭐️
This package helps us run unit tests. One requirement to keep in mind is that your python file containing the unit tests should begin with 'test_'.
Only assert statements are supported. To install the package
pip install pytest
To run the unit tests, type the following commands
pytest test_helpers.py
If all your tests pass, you should see a similar output
test_helpers.py ....             [100%]

========= 4 passed in 0.01s ===========
If you get an error related to multiple relative imports
astroid.exceptions.TooManyLevelsError:
It is probably an issue with one of pytest's dependencies. You'll have to uninstall astroid and install it again. This ensures the altest astroid version is installed.
pip uninstall astroid
pip install astroid
After this, we will have to uninstall pytest and install pytest
pip uninstall pytest
pip install pytest
Formatting ✍️
YAPF 12k+ ⭐️
This was developed by google and supports in-place formatting. To install the package
pip install yapf
To format your files, type the following
yapf --in-place *.py
This will format all your top level python files, if you want to include folders as well you can use the following
yapf --in-place **/*.py
However, this will also include our virtual environment folder. To ignore the venv folder, simply create a file .yapfignore and add venv to it.
Note: This command might take some time to run. Instead of '**' you could use the folder's specific names.
isort 4.1k+ ⭐️
This packages sorts your import statements to ensure they follow pep8 rules.
Imports should be grouped in the following order:
  • Standard library imports.
  • Related third party imports.
  • Local application/library specific imports.
  • isort re-orders import statements to ensure the above rule is followed.
    To install the package
    pip install isort
    To run isort
    isort .
    autoflake 400+⭐️
    It helps in getting rid of unused imports, variables and object keys.
    To install the package
    pip install autoflake
    To run autoflake
    autoflake --in-place --remove-unused-variables --remove-all-unused-imports *.py
    Some other formatters
  • autopep8 3.9k+ ⭐️
  • black 22.1k+ ⭐️
  • Linting 🔎
    Pylint 3.5k+ ⭐️
    pylint ensures your code is following pep8 rules and standards. It gives each python file a score out of 10 (It can given you a negative score as well)
    To install the package
    pip install pylint
    To run the linter
    pylint --fail-under=7 *.py
    The argument --fail-under is the lower bound, if any file has a score below the lower bound, an error will be returned.
    Pre-commit Hook 🪝
    What Are Git Hooks?
    Git hooks are basically scripts fired before an important action occurs, e.g., before a commit is made, before code is pushed to a repo after a commit is made, etc. You can learn more about Git Hooks and the different kinds of hooks over here.
    We will be focussing on a pre-commit hook.A pre-commit hook is a hook that is run before you make a commit.
    First let's install the package
    pip install pre-commit
    Now we will generate a sample pre-commit hook YAML file, we will edit this later.
    pre-commit sample-config
    Now let's add our hook
    pre-commit install
    Now before every commit, the pre-commit hook defined in our YAML file will be executed.
    Now let's update our YAML file.
    Remove everything and only keep the following
    repos:
        - repo: local
          hooks:
    We will add our plugins(packages) under hooks: in the YAML file. Below is the general syntax for the plugin
    - id: (unique id of hook)
         name: (name to be displayed in terminal)
         entry: (command to excute)
         language: system (for our case, always system) 
         always_run: true (if true, it will always run)
         pass_filenames: true (if true, hook will have access to the file name)
    Let's define a sample plugin for YAPF
    - id: YAPF 
         name: YAPF 🧹
         entry: zsh -c 'yapf --in-place *.py'
         language: system
         always_run: true
         pass_filenames: true
    If you are using bash or are on windows, replace the zsh in 'entry' with bash.
    All the other plugins are pretty similar, below is the entire YAML file with all the plugins
    repos:
    - repo: local
    hooks:
    - id: YAPF
    name: YAPF 🧹
    entry: zsh -c 'yapf --in-place *.py'
    language: system
    always_run: true
    pass_filenames: true
    - id: isort
    name: isort 📚
    entry: zsh -c 'isort .'
    language: system
    always_run: true
    pass_filenames: true
    - id: autoflake
    name: autoflake ❄️
    entry: zsh -c 'autoflake --in-place --remove-unused-variables --remove-all-unused-imports *.py'
    language: system
    always_run: true
    pass_filenames: true
    - id: pylint
    name: pylint 🔎
    entry: zsh -c 'pylint --fail-under=7 *.py'
    language: system
    always_run: true
    pass_filenames: true
    - id: pytest
    name: Unit Tests 🧪
    entry: zsh -c 'pytest test_helpers.py'
    language: system
    always_run: true
    pass_filenames: false

    Whenever you update your YAML file, you will have to add the file to the staging area using git add . or git add .pre-commit-config.yaml

    Below is a successful commit
    Conclusion
    Setting up a pre-commit hook will ensure your code follows pep8 standards and is properly formatted.
    I hope you found the article useful. Add me on LinkedIn, Twitter

    24

    This website collects cookies to deliver better user experience

    Automate Clean Code and Linting in Python