Factory Fixtures

Factory fixtures, what in the whole earth would that be? When you want to test your code thoroughly, you have to write quite a few tests. This way, you ensure that your code is working as expected. Nevertheless, you might face the following problems:

• A simple change that is not related to existing tests might break a lot of tests (e.g., a new required attribute is added to an object).
• Tests are unreadable due to a lot of setup code (e.g., only one attribute is relevant for the test, but you have to create a complete object with 5 attributes).

We can solve these problems by using factory fixtures. Factory fixtures are fixtures that return functions to create objects – as opposed to regular fixtures that return already created objects. Inside them, we can apply sane defaults and override only the relevant attributes for the test. Enough talking; let’s take a look at the examples.

Password validation

Let’s say we have a user model with the following password requirements:

• At least 8 characters
• At least one uppercase letter
• At least one lowercase letter
• At least one special character
• At least one digit

The user model might look like this:

from uuid import UUID

import bcrypt
from pydantic import BaseModel, field_validator, Field, EmailStr

class User(BaseModel):
    id: UUID
    name: str
    email: EmailStr
    hashed_password: str = Field(..., alias="password")

    @field_validator("hashed_password", mode="before")
    @classmethod
    def validate_password(cls, value):
        if len(value) < 8:
            raise ValueError("password must be at least 8 characters")

        if not any(char.isupper() for char in value):
            raise ValueError("password must have at least one uppercase letter")

        if not any(char.islower() for char in value):
            raise ValueError("password must have at least one lowercase letter")

        if not any(char in "!@#$%^&*()" for char in value):
            raise ValueError("password must have at least one special character: !@#$%^&*()")

        if not any(char.isdigit() for char in value):
            raise ValueError("password must have at least one number")

        return bcrypt.hashpw(value.encode(), bcrypt.gensalt()).decode()

Tests without factory fixtures might look like this:

from uuid import uuid4

import pytest

from models import User

def test_password_must_be_at_least_8_characters():
    with pytest.raises(ValueError):
        User(id=uuid4(), name="John Doe", email="john@doe.com", password="Sec123!")

def test_password_must_be_have_at_least_one_uppercase_letter():
    with pytest.raises(ValueError):
        User(id=uuid4(), name="John Doe", email="john@doe.com", password="secret123!")

def test_password_must_have_at_least_one_lowercase_letter():
    with pytest.raises(ValueError):
        User(id=uuid4(), name="John Doe", email="john@doe.com", password="SECRET123!")

def test_password_must_have_at_least_one_special_character():
    with pytest.raises(ValueError):
        User(id=uuid4(), name="John Doe", email="john@doe.com", password="Secret123")

def test_password_must_have_at_least_one_number():
    with pytest.raises(ValueError):
        User(id=uuid4(), name="John Doe", email="john@doe.com", password="SecretSecret!")

def test_user_created_with_valid_password():
    assert User(id=uuid4(), name="John Doe", email="john@doe.com", password="Secret123!").hashed_password

Tests with factory fixtures might look like this:

from typing import Callable
from uuid import uuid4

import pytest

from models import User

@pytest.fixture
def create_user() -> Callable[..., User]:
    def _create_user(**kwargs) -> User:
        user = User(
            id=kwargs.get("id", uuid4()),
            name=kwargs.get("name", "John Doe"),
            email=kwargs.get("email", "john@doe.com"),
            password=kwargs.get("password", "Secret123!")
        )
        return user
    return _create_user

def test_password_must_be_at_least_8_characters(create_user: Callable[..., User]):
    with pytest.raises(ValueError):
        create_user(password="Sec123!")

def test_password_must_be_have_at_least_one_uppercase_letter(create_user: Callable[..., User]):
    with pytest.raises(ValueError):
        create_user(password="secret123!")

def test_password_must_have_at_least_one_lowercase_letter(create_user: Callable[..., User]):
    with pytest.raises(ValueError):
        create_user(password="SECRET123!")

def test_password_must_have_at_least_one_special_character(create_user: Callable[..., User]):
    with pytest.raises(ValueError):
        create_user(password="Secret123")

def test_password_must_have_at_least_one_number(create_user: Callable[..., User]):
    with pytest.raises(ValueError):
        create_user(password="SecretSecret!")

def test_user_created_with_valid_password(create_user: Callable[..., User]):
    assert create_user(password="Secret123!").hashed_password

Before we compare the two examples, let’s refresh our memory of what pytest fixtures are. pytest fixtures are decorated functions that provide data/objects/… to our tests. They are executed before the test function by pytest. Their result is passed to the test function as an argument. We can reference fixtures in our tests by adding arguments with matching names to the test functions.

Readability Is Improved With Factory Fixtures

Let’s now compare the readability of the two examples above. In both cases, we’re testing only password validation. To cover all of our requirements, we have to write 6 tests. That’s the same for both cases. Anyhow, there are some differences to mention.

As you can see, the first example requires us to create a user object with all the required data inside our test cases. That’s required even though password is the only relevant value for the tests. This hurts the readability of our tests.

On the other hand, the second example uses a factory fixture to create user objects. Using factory fixtures, we clearly communicate that the only relevant thing for our tests is the password. This makes our tests much more readable. Especially, when the model has more attributes. This means that we need to write less code while improving the readability of our tests. That’s a perfect win-win situation.

Factory fixtures improve resistance to unrelated changes

As we’ve seen, readability is improved by using factory fixtures. Not only that, but our tests are also more resistant to unrelated changes. Let’s say we add a new required attribute to the user model – date_of_birth. What will happen with our tests?

In the first example, we need to go and update all of our tests to include the new attribute. This is not a big deal for 6 tests, but what if we have 1000 tests that depend on the user model? This becomes a nightmare. A simple change in implementation, a significant change in tests. That’s not what we want.

Looking at the example using factory fixtures, we can see that only one change is required. We need to set additional attributes inside the factory fixture. That’s it. We don’t need to touch any of our tests. A simple change in implementation, a simple change in tests. That’s what we want.

See example:

from uuid import UUID

import bcrypt
from pydantic import BaseModel, field_validator, Field, EmailStr

class User(BaseModel):
    id: UUID
    name: str
    email: EmailStr
    hashed_password: str = Field(..., alias="password")
    status: str
    

    @field_validator("hashed_password", mode="before")
    @classmethod
    def validate_password(cls, value):
        if len(value) < 8:
            raise ValueError("password must be at least 8 characters")

        if not any(char.isupper() for char in value):
            raise ValueError("password must have at least one uppercase letter")

        if not any(char.islower() for char in value):
            raise ValueError("password must have at least one lowercase letter")

        if not any(char in "!@#$%^&*()" for char in value):
            raise ValueError("password must have at least one special character: !@#$%^&*()")

        if not any(char.isdigit() for char in value):
            raise ValueError("password must have at least one number")

        return bcrypt.hashpw(value.encode(), bcrypt.gensalt()).decode()

Change in tests with factory fixtures:

from typing import Callable
from uuid import uuid4

import pytest

from models import User

@pytest.fixture
def create_user() -> Callable[..., User]:
    def _create_user(**kwargs) -> User:
        user = User(
            id=kwargs.get("id", uuid4()),
            name=kwargs.get("name", "John Doe"),
            email=kwargs.get("email", "john@doe.com"),
            password=kwargs.get("password", "Secret123!"),
            status=kwargs.get("status", "active")  # NEW ATTRIBUTE
        )
        return user
    return _create_user

def test_password_must_be_at_least_8_characters(create_user: Callable[..., User]):
    with pytest.raises(ValueError):
        create_user(password="Sec123!")

def test_password_must_be_have_at_least_one_uppercase_letter(create_user: Callable[..., User]):
    with pytest.raises(ValueError):
        create_user(password="secret123!")

def test_password_must_have_at_least_one_lowercase_letter(create_user: Callable[..., User]):
    with pytest.raises(ValueError):
        create_user(password="SECRET123!")

def test_password_must_have_at_least_one_special_character(create_user: Callable[..., User]):
    with pytest.raises(ValueError):
        create_user(password="Secret123")

def test_password_must_have_at_least_one_number(create_user: Callable[..., User]):
    with pytest.raises(ValueError):
        create_user(password="SecretSecret!")

def test_user_created_with_valid_password(create_user: Callable[..., User]):
    assert create_user(password="Secret123!").hashed_password

This way, we don’t need to touch our password tests when a new (non-password-related) attribute is added to the user model. We would still need to touch them if password validation logic changes, but that’s expected.

Conclusion

As you can see, factory fixtures are a great way to improve our tests’ readability and resistance to unrelated changes. They might require a bit more setup, but they pay off in the long run.

You can find code examples in this Github repository.