How To Test External Dependencies With Pytest, Docker, and Tox (2024)

When developing a new application, we often need external services such as databases, message brokers, cache memory, or APIs to other micro-services. Testing the interfaces between these components is a practice that is often neglected. Some completely skip these tests, and others mock responses. This, however, leaves integration tests incomplete, as those interfaces can easily fail due to changes in vendors' APIs, database schema changes, and configuration issues.

One way to conduct these tests is by running the dependencies as containers that communicate with the application in an environment similar to production. In this article, I illustrate how to run these tests in a CI pipeline using GitHub actions. To do this, you must write a small Python application with access to a postgres database to create and select users.

The tests will be written using pytest, one of the leading testing frameworks for Python. The database will be run as a container using Docker. Tests will be coordinated with tox, a testing orchestrator for Python, which originally emerged as a tool to test different Python versions, and has evolved to allow developers to isolate testing from development environments, centralise testing configuration, and even coordinate dependencies using docker containers.

You can follow the instructions and code snippets or check out the whole (but small) project directly on my GitHub repo. The file structure of the project is this one:

.
├── README.md
├── migrations
│ └── schema.sql
├── requirements.txt
├── src
│ └── testing_containers
│ ├── __init__.py
│ ├── db
│ │ ├── __init__.py
│ │ └── db.py
│ └── model
│ ├── __init__.py
│ └── users.py
├── tests
│ ├── __init__.py
│ ├── conftest.py
│ └── db
│ └── test_db.py
└── tox.ini

Since we are using a Postgres database as our dependency, let’s start by writing a small table representing Users in our application. I’ll call it migrations/schema.sql. Here’s what it looks like:

CREATE TABLE users (
 email varchar(64) primary key,
 name varchar(64) not null
);

To represent the entities in the application, I will use pydantic . This is a great way to represent objects, get type hints, and it is especially useful to perform data validations when we use these entities in APIs. So, create a new Python file, call it users.py, and paste the following:

from pydantic import BaseModel
class User(BaseModel):
 """Representation of User entity"""
 name: str
 email: str

Now we can write some boilerplate for our database operations. Let’s start with some functions that wrap our postgres driver ( psycopg2 in this case). As the intent of this article is not to teach how to use this driver, I will not elaborate on them, but I will encourage you to read the PYnative tutorial. We can start our class like this:

from typing import List
import psycopg2
from psycopg2.extras import execute_values
from testing_containers.model.users import User
class Repo:
 # psycopg2 wrapper
 def __init__(self) -> None:
 self.conn = None
 try:
 self.connect()
 logging.info("db: database ready")
 except Exception as err:
 logging.error("db: failed to connect to database: %s", str(err))
 self.close()
 raise err
 def connect(self):
 """Stores a connection object `conn` of a postgres database"""
 logging.info("db: connecting to database")
 conn_str = os.environ.get("DB_DSN")
 self.conn = psycopg2.connect(conn_str)
 def close(self):
 """Closes the connection object `conn`"""
 if self.conn is not None:
 logging.info("db: closing database")
 self.conn.close()
 def execute_select_query(self, query: str, args: tuple = ()) -> List[tuple]:
 """Executes a read query and returns the result
 Args:
 query (str): the query to execute.
 args (tuple, optional): arguments to the select statement. Defaults to ().
 Returns:
 List[tuple]: result of the select statement. One element per record
 """
 with self.conn.cursor() as cur:
 cur.execute(query, args)
 return list(cur)
 def execute_multiple_insert_query(self, query: str, data: List[tuple], page_size: int = 100) -> None:
 """Execute a statement using :query:`VALUES` with a sequence of parameters.
 Args:
 query (str): the query to execute. It must contain a single ``%s``
 placeholder, which will be replaced by a `VALUES list`__.
 Example: ``"INSERT INTO table (id, f1, f2) VALUES %s"``.
 data (List[tuple]): sequence of sequences or dictionaries with the arguments to send to the query.
 page_size (int, optional): maximum number of *data* items to include in every statement.
 If there are more items the function will execute more than one statement. Defaults to 100.
 """
 with self.conn.cursor() as cur:
 execute_values(cur, query, data, page_size=page_size)

Now we write some functions to:

1. Retrieve a user by name
2. Create new user(s)

def get_user(self, name: str) -> (User | None):
 """Retrieve a User by a name"""
 query = "SELECT name, email FROM users WHERE name = %s"
 res = self.execute_select_query(query, (name,))
 if len(res) == 0:
 return None
 record = res[0]
 return User(name=record[0], email=record[1])
def insert_users(self, users: List[User]) -> None:
 """Given a list of Users, insert them in the db"""
 query = "INSERT INTO users (name, email) VALUES %s"
 data: List[tuple] = [(u.name, u.email) for u in users]
 self.execute_multiple_insert_query(query, data)

Awesome! We have some functions that interact without business entities.

Notice that if we don’t test these functions, we are susceptible to errors if the schema changes or if we violate database constraints.

So, let’s start testing!

import pytest
from psycopg2.errors import UniqueViolation
@pytest.mark.usefixtures("repo")
class TestRepo:
 def test_insert_users(self):
 repo: Repo = self.repo # repository instanced passed by the fixture
 # define some users
 alice = User(name="alice", email="alice@example.com")
 bob = User(name="bob", email="bob@example.com")
 robert = User(name="robert", email="bob@example.com")
 # check that the users are not there at first
 result = repo.get_user("alice")
 assert result is None
 # check that the users are there after inserting
 users = [alice, bob]
 repo.insert_users(users)
 result = repo.get_user("alice")
 assert result == alice
 result = repo.get_user("bob")
 assert result == bob
 # check that pk fails
 with pytest.raises(UniqueViolation):
 repo.insert_users([robert])

The first line is a decorator that specifies that our test class TestRepo will use a fixture called repo. Fixtures are baseline functions that allow us to produce consistent and repeatable test results.

Let’s understand this line repo: Repo = self.repo. We are creating a variable called repo of type Repo which is our previously defined repository class. It is being reassigned from self.repo, which is coming from the fixture, as I will explain later.

We assert a couple of things:

1. There are no users at the beginning of the test
2. Alice and Bob users are retrieved after insertion
3. There is a UniqueViolation exception, and it is raised when we attempt to create a new user with an already existing email.

Let’s write our fixture in a new conftest.py file.

@pytest.fixture(scope="class", name="repo")
def repo(request):
 """Instantiates a database object"""
 db = Repo()
 try:
 request.cls.repo = db
 yield db
 finally:
 db.close()

In this fixture, we do the following:

1. Instantiate our repository class db = Repo .
2. Within a try block (as the test execution might raise an exception), we use pytest’s request fixture so our test class can access the repo request.cls.repo = db . I encourage you to learn more about the request fixture!
3. Then we yield the instance to be used in the test functions.
   Within a finally block, we make sure we close the database connection.

Great! We have our tests, but if you are eager to run pytest already, you will see that the tests fail at the fixture, as we cannot instantiate the repository without a postgres database!

Here is where tox comes to the rescue. Create a new file called tox.ini:

[tox]
envlist = py310
[testenv]
setenv =
 DB_DSN = postgres://postgres:postgres@localhost:5432/postgres?sslmode=disable
deps =
 -r requirements.txt
commands = pytest

This minimal configuration file tells us we will perform testing with a Python 3.10 environment. It sets an environmental variable DB_DSN, specifies a requirement file to be installed in a virtual environment, and calls pytest. My requirements file looks like this:

psycopg2-binary
pydantic
pytest

By the way, I recommend using pip-toos to pin dependencies. That is out of the scope of this tutorial, but you can read it here: https://github.com/jazzband/pip-tools.

Running your tests now is as easy as just installing and running tox:

python -m pip install — user tox

tox

Of course, this fails because I have been promising that tox will coordinate a postgres container, and I haven’t done so.

Tox is a useful tool that can get powerful with its plugins. Tox-docker is one of them, and it’s easily installed by running the following command:

pip install tox-docker

Now we can extend our tox.ini. Here’s how to do that:

[tox]
envlist = py310
[testenv]
setenv =
 DB_DSN = postgres://postgres:postgres@localhost:5432/postgres?sslmode=disable
deps =
 -r requirements.txt
commands = pytest
docker = postgres
[docker:postgres]
image = postgres:13.4-alpine
environment =
 POSTGRES_DB=postgres
 PGUSER=postgres
 POSTGRES_PASSWORD=postgres
 POSTGRES_HOST_AUTH_METHOD=trust
ports =
 5432:5432/tcp
healthcheck_cmd = pg_isready
healthcheck_timeout = 5
healthcheck_retries = 5
healthcheck_interval = 5
healthcheck_start_period = 1
volumes =
 bind:ro:{toxinidir}/migrations/schema.sql:/docker-entrypoint-initdb.d/0.init.sql

Notice that in our testenv section, we specify we will use a docker container named postgres which we immediately after define. We set the docker image it should (pull) use, environmental variables, ports to map, health checks (useful to make sure our tests are running only when our containers are healthy), and volumes (notice I refer tomigrations/schema.sql which contains our SQL table definition). Please check tox-docker documentation if you want more details.

Now, by running tox we pass our tests!

Notice that tox creates the containers, runs the tests, and then removes the containers for us. Pretty cool, right?

Running our tests locally is a great practice, but to be very thorough, it is better if we also run them in our version control tool when making a pull request. With GitHub Actions, we can create a small CI pipeline to run tox every time a commit is pushed to a pull request. Just create this file, /.github/worflows/pr-test.yaml :

name: PR Test
on:
 pull_request:
 branches:
 - main
jobs: 
 test:
 runs-on: ubuntu-latest
 strategy:
 matrix:
 python-version: ["3.10"]
 name: With Python ${{ matrix.python-version }}
 steps:
 - name: Checkout code
 uses: actions/checkout@v2
 - name: Setup python
 uses: actions/setup-python@v2
 with:
 python-version: ${{ matrix.python-version }}
 architecture: x64
 - name: Install dependencies
 run: |
 python -m pip install --upgrade pip
 python -m pip install tox tox-gh-actions
 - name: Test with tox
 run: tox

Now, every time we create a pull request to our code base, GitHub Actions will run tox and make sure our tests pass.

With the current approach, we can easily scale dependency testing by adding more docker containers in the tox configuration.

[docker:redis]
image = bitnami/redis:latest
environment =
 ALLOW_EMPTY_PASSWORD=yes
 REDIS_PORT_NUMBER=7000
ports =
 7000:7000/tcp
healthcheck_cmd = redis-cli ping
healthcheck_timeout = 5
healthcheck_retries = 5
healthcheck_interval = 5
healthcheck_start_period = 1

External dependencies that can’t be containerized or that represent costs or protected resources, such as a private API, should better be mocked.
If the usefulness does not convince you of tox, a different approach is to set the dependencies in a docker-compose file, create and run the services, wait for them to be healthy, run the pytests, and gracefully stop and remove containers.

1. GitHub repo for this project: https://github.com/vrgsdaniel/testing-containers
2. Pytest documentation: https://docs.pytest.org/en/7.2.x/
3. Pytest fixtures: https://docs.pytest.org/en/6.2.x/fixture.html
4. Pytest request fixture: https://docs.pytest.org/en/6.2.x/reference.html#std-fixture-request
5. Tox: https://tox.wiki/en/latest/
6. Tox-docker: https://tox-docker.readthedocs.io/en/latest/
7. GitHub Actions: https://github.com/features/actions
8. PYnative postgres tutorial: https://pynative.com/python-postgresql-tutorial/
9. Pip-tools: https://github.com/jazzband/pip-tools

Want to Connect?
If you found my post interesting, you can visit my LinkedIn or GitHub :).