GitLab : Automatically testing your Python project

Whatever programming language your are using for your project, GitLab continuous integration system (gitlab-ci) is a fantastic tool that allows you to automatically run tests when code is pushed to your repository. Java, PHP, Go, Python or even LaTeX, no limit here! In this blog post we review a few examples for the Python programming language.

In GitLab, the different tests are called jobs. These jobs are executed by a gitlab-runner, that can be installed on the same server as you main GitLab, or on one or multiple separate servers.

Different options exist to run the jobs: using VirtualBox virtual machines, using Docker containers or using no virtualization at all. Most administrators configure gitlab-runner to run jobs using Docker images. This is also what we will assume for the remainder of this post.

PyLint

Pylint is a powerful tool that performs static code analysis and checks that your code is written following Python coding standards (called PEP8).

To automatically check your code when you push to your repository, add a file called .gitlab-ci.yml at the root of your project with following content:

test:pylint:
    image: python:3.6
    script:
        - pip install pylint --quiet
        - pylint --ignored-classes=_socketobject *.py

test:pylint is simply the name of the job. You can choose whatever you want. The rest of the code indicates that gitlab-runner should use the docker image python:3.6, and run the mentioned commands.

When using some external classes (like sockets), PyLint may complain that the used method does not exist, although the method does actually exist. You can ignore these errors by appending --ignored-classes=... to the pylint command line.

You can also specify a directory (instead of *.py), but it must be a Python module and include __init__.py.

Good to know: Pylint is shipped with Pyreverse which creates UML diagrams from python code.

Unit testing wity PyTest

PyTest is a framework designed to help you test your Python code. Here is an example of a test:

# content of test_sample.py
def func(x):
    return x + 1

def test_answer():
    assert func(3) == 5

To run these tests automatically when you push code to your repository, add the following job to your .gitlab-ci.yml.

test:pytest:
  image: python:3.6
  script:
    - pip install pytest --quiet
    - pytest

pytest will automatically discover all test files in your project (all files named test_*.py or *_test.py) and execute them.

Testing multiple versions of Python

One of the main benefits of automatic testing with gitlab-ci is that your can test multiple versions of Python at the same time. There is however one caveat: if you plan to test your code with both Python 2 and Python 3, you will at least need to disable the superfluous-parens test in Python 2 (for the print statement, that became a function in Python 3). Here is a full example of .gitlab-ci.yml :

test:pylint:36:
    image: python:3.6
    script:
        - pip install pylint --quiet
        - pylint --ignored-classes=_socketobject *.py

test:pytest:36:
  image: python:3.6
  script:
    - pip install pytest --quiet
    - pytest

test:pylint:27:
    image: python:2.7
    script:
        - pip install pylint --quiet
        - pylint --disable superfluous-parens --ignored-classes=_socketobject *.py

test:pytest:27:
  image: python:2.7
  script:
    - pip install pytest --quiet
    - pytest

This blog post is licensed under CC BY-SA 4.0