Testing behavior, not implementation details – part 3

One of the properties of a valuable test is that it’s fast. While tests with a database might be fast enough for simple projects, their execution times quickly add up. As mentioned in “Running tests in parallel with pytest”, you can run tests in parallel. Nevertheless, even that has its limits. To run more and more tests in the same time frame, you need more and more CPU cores or machines. There’s still your machine. At some point, it will take too much time to run all the tests locally. So you’ll probably skip them. So the question is – can we make our tests for business logic faster? Yes, we can! We can do that by implementing test doubles using contract tests. So, let’s look into that.

This tip is part of Test behavior, not implementation details series

1. Test behavior, not implementation details – part 1
2. Test behavior, not implementation details – part 2
   
3. Test behavior, not implementation details – part 3
   

Fast vs. Durable

In “Testing Behavior, not implementation details – part 1”, we’ve implemented two stores – an in-memory one and SQLite one. We’ve shown with tests that they both expose the same observable behavior. Can we leverage this somehow? Of course, we can. And we should. We can use the in-memory implementation in tests and SQLite one on live server. Why would we do that? In tests, we want to use the fastest implementation – to speed our tests up. When deployed on the live server, we want the data to be persisted between different deployments and server restarts. That’s why we have to use SQLite implementation. Nevertheless, to really do that we need to make sure that both stores expose the same observable behavior. So how do we do that?

Contract tests

In part 1, we’ve seen that we can use the same tests for both implementations. Yet, the tests were written twice. Having tests written twice decreases our confidence in the fact that both implementations are doing the same. This means that we might change tests for one while forgetting about the tests for the other. This can quickly lead to inconsistencies between implementations. We want to be confident that both implementations behave the same way. So we can switch between them based on our needs. That’s where contract tests come into play. We can use them to execute precisely the same set of tests for different implementations. This way, we ensure that implementations always expose the same observable behavior. So, let’s take a look at an example.

# models.py
from dataclasses import dataclass
from enum import Enum

class TaskStatus(str, Enum):
    OPEN = "OPEN"
    CLOSED = "CLOSED"

@dataclass
class Task:
    title: str
    status: TaskStatus
    owner: str

# store.py
import sqlite3
from abc import ABC

from models import Task, TaskStatus

class TaskStore(ABC):
    def add_task(self, task: Task) -> None:
        raise NotImplementedError

    def list_open(self, owner: str) -> list[Task]:
        raise NotImplementedError

class TaskStoreSQLite(TaskStore):
    def __init__(self, connection: sqlite3.Connection):
        self._connection = connection

    def add_task(self, task: Task) -> None:
        self._connection.execute(
            "INSERT INTO tasks (title, status, owner) VALUES (?, ?, ?)",
            (task.title, task.status.name, task.owner),
        )

    def list_open(self, owner: str) -> list[Task]:
        cursor = self._connection.execute(
            "SELECT title, status, owner FROM tasks WHERE status = 'OPEN' AND owner = ?",
            (owner,),
        )
        return [
            Task(title, TaskStatus(status), owner)
            for title, status, owner in cursor.fetchall()
        ]

class TaskStoreInMemory(TaskStore):
    def __init__(self):
        self._tasks = []

    def add_task(self, task: Task) -> None:
        self._tasks.append(task)

    def list_open(self, owner: str) -> list[Task]:
        return [
            task
            for task in self._tasks
            if task.status == TaskStatus.OPEN and task.owner == owner
        ]

# tests.py
import sqlite3

import pytest

from models import Task, TaskStatus
from store import TaskStoreSQLite, TaskStore, TaskStoreInMemory

class TaskStoreContract:
    @pytest.fixture
    def store(self) -> TaskStore:
        raise NotImplementedError

    def test_added_task_listed(self, store: TaskStore):
        task = Task(title="Do the dishes", status=TaskStatus.OPEN, owner="johndoe")

        store.add_task(task)

        assert store.list_open(owner=task.owner) == [task]

    def test_closed_task_not_listed_in_open_tasks(self, store: TaskStore):
        task = Task(title="Do the dishes", status=TaskStatus.CLOSED, owner="johndoe")

        store.add_task(task)

        assert store.list_open(owner=task.owner) == []

    def test_task_from_other_owner_not_listed_in_open_tasks(self, store: TaskStore):
        task = Task(title="Do the dishes", status=TaskStatus.OPEN, owner="johndoe")

        store.add_task(task)

        assert store.list_open(owner="anotherowner") == []

class TestTaskStoreSQLite(TaskStoreContract):
    @pytest.fixture
    def connection(self, tmp_path) -> sqlite3.Connection:
        connection = sqlite3.connect(tmp_path / "test.db")
        connection.execute("CREATE TABLE tasks (title TEXT, status TEXT, owner TEXT)")
        return connection

    @pytest.fixture
    def store(self, connection) -> TaskStore:
        return TaskStoreSQLite(connection)

class TestTaskStoreInMemory(TaskStoreContract):
    @pytest.fixture
    def store(self) -> TaskStore:
        return TaskStoreInMemory()

First of all, to do the contract testing, both implementations must expose the same interface. To do that, we must really hide the implementation details. Once we ensure that, we can go to tests. So what do we do? We define the contract – a set of tests – on a class. In this particular case, that’s TaskStoreContract. On the contract level, we define the required fixture for an object that we’re testing, but we don’t implement it. In our example, that’s a store fixture. Then, we use the contract class to create a test class for each implementation. In our case, these are TestTaskStoreSQLite and TestTaskStoreInMemory. These classes implement only the fixture specified by the contract. As you can see, the setup for this fixture isn’t the same. SQLite implementation needs a database connection, while in-memory doesn’t. That’s expected. The important part is that common behavior (after the object is created) satisfies exactly the same set of tests.

Conclusion

Using contract tests, we can effectively implement test doubles. This way, we ensure the same observable behavior for different implementations. For example, this way we can implement in-memory store that can be used in tests as a test double for SQLite implementation. This way we can test our business logic with much faster tests. Also, contract tests force use to really hide the implementation details which improves the quality of our code and tests.

You can find full code examples in this Github repository.