create_autospec – easily ensure that mocked objects are called correctly

When writing automated tests, we aim towards fast & reliable tests. Many times we need to use test doubles to ensure that. In Python, we quite often use MagicMock to replace objects during tests. It comes very handy, but its usage can also backfire very quickly. Mostly because we can call it in the most stupid way possible, but it still won’t produce an error. Let’s take a look at how to improve that using create_autospec.

Standard library to the rescue

As on many other occasions, the Python standard library covers us. Inside the unittest.mock module, we can find the create_autospec function. It takes an object as a spec and creates a mock object with the same attributes as the provided spec. In other words, it creates an instance of an object with the same properties and methods as the provided one. For example, if you provide a class Carwith methods start and stop, you’ll get an instance of mock with these two methods defined. Now, if you use that instance and try to call the foo method, an exception will be raised. So how does this help us when testing? Let’s take a look!

Stubbing with create_autospec

We know a couple of different test doubles – dummies, fakes, stubs, spies, and mocks. (You can learn more here). create_autospec is very useful when replacing objects with stubs. That’s when we use predefined responses to certain calls. That’s useful when we want to replace some read-like object. For example, an ML model that’s predicting sentiment. Using create_autospec, we can make sure our code integrates the model and actually calls it correctly. By using only MagicMock, we could make a typo in method name when calling it or provide wrong arguments (e.g., one argument too few), but the test would still pass. With create_autospec, the exception is raised in both cases. Some of these errors can be caught by type checkers. Anyhow, no one likes a situation where all the tests are passing, but the code is actually broken, right? This happens quite regularly when you combine the usage of MagicMock and auto-refactoring. When using create_autospec, we’ll have a failing test if the implementation uses an object incorrectly. See the examples below:

# test_stubs.py
import time
from unittest.mock import MagicMock, create_autospec

class SentimentClassifier:
    def predict(self, text: str) -> str:
        # Imagine that prediction is taking a long time
        time.sleep(5)
        return "positive"

class EnrichBlogPost:
    def __init__(self, sentiment_classifier: SentimentClassifier) -> None:
        self._sentiment_classifier = sentiment_classifier

    def execute(self, blog_post: dict) -> dict:
        sentiment = self._sentiment_classifier.predict(blog_post["text"])
        blog_post["sentiment"] = sentiment
        return blog_post

class EnrichBlogPostWrongUsage:
    def __init__(self, sentiment_classifier: SentimentClassifier) -> None:
        self._sentiment_classifier = sentiment_classifier

    def execute(self, blog_post: dict) -> dict:
        sentiment = self._sentiment_classifier.classify(blog_post["text"])
        blog_post["sentiment"] = sentiment
        return blog_post

def test_blog_post_enriched_with_sentiment_magic_mock():
    sentiment_classifier = MagicMock()
    sentiment_classifier.predict.return_value = "positive"
    enrich_blog_post = EnrichBlogPost(sentiment_classifier=sentiment_classifier)
    blog_post = {"text": "I love Python"}

    enriched_blog_post = enrich_blog_post.execute(blog_post=blog_post)

    assert enriched_blog_post["sentiment"] == "positive"
    # This one is passes - usage and setup are correct

def test_blog_post_enriched_with_sentiment_create_autospec():
    sentiment_classifier = create_autospec(SentimentClassifier)
    sentiment_classifier.predict.return_value = "positive"
    enrich_blog_post = EnrichBlogPost(sentiment_classifier=sentiment_classifier)
    blog_post = {"text": "I love Python"}

    enriched_blog_post = enrich_blog_post.execute(blog_post=blog_post)

    assert enriched_blog_post["sentiment"] == "positive"
    # This one passes - usage and setup are correct

def test_blog_post_enriched_with_sentiment_wrong_usage_magic_mock():
    sentiment_classifier = MagicMock()
    sentiment_classifier.classify.return_value = "positive"
    enrich_blog_post = EnrichBlogPostWrongUsage(sentiment_classifier=sentiment_classifier)
    blog_post = {"text": "I love Python"}

    enriched_blog_post = enrich_blog_post.execute(blog_post=blog_post)

    assert enriched_blog_post["sentiment"] == "positive"
    # This one passes because test setup and usage are both wrong, but it should fail

def test_blog_post_enriched_with_sentiment_wrong_usage_create_autospec():
    sentiment_classifier = create_autospec(SentimentClassifier)
    sentiment_classifier.classify.return_value = "positive"
    enrich_blog_post = EnrichBlogPostWrongUsage(sentiment_classifier=sentiment_classifier)
    blog_post = {"text": "I love Python"}

    enriched_blog_post = enrich_blog_post.execute(blog_post=blog_post)

    assert enriched_blog_post["sentiment"] == "positive"
    # AttributeError: Mock object has no attribute 'classify'
    # This one fails because we're trying to use 'classify' which isn't present in SentimentClassifier

Spying with create_autospec

The usage can be quite similar when we use spies instead of stubs. That’s when we assert that something was called in a certain way. That comes in handy when sending some data somewhere out of our current process – for example, when we need to send an email or when syncing user data to CRM. Similar to stubs, create_autospec makes sure that tests fail if the usage of mocked objects doesn’t match the expectations. See example below:

# test_spies.py
from unittest.mock import MagicMock, create_autospec

class MyCRMService:
    def sync_user(self, *, first_name: str, last_name: str, email: str) -> None:
        print("Calling MyCRM's REST API to send data there. That's slow and unreliable.")

class CreateUser:
    def __init__(self, *, crm_service: MyCRMService) -> None:
        self._crm_service = crm_service

    def execute(self, *, first_name: str, last_name: str, email: str) -> None:
        print("Store user in database")
        print("Send welcome email")
        self._crm_service.sync_user(first_name=first_name, last_name=last_name, email=email)

class CreateUserWrongUsage:
    def __init__(self, *, crm_service: MyCRMService) -> None:
        self._crm_service = crm_service

    def execute(self, *, first_name: str, last_name: str, email: str) -> None:
        print("Store user in database")
        print("Send welcome email")
        self._crm_service.sync_user(first_name, last_name, email)

def test_user_data_synced_to_crm_magic_mock():
    crm_service = MagicMock()
    create_user = CreateUser(crm_service=crm_service)
    first_name = "John"
    last_name = "Doe"
    email = "john@doe.com"

    create_user.execute(first_name=first_name, last_name=last_name, email=email)

    crm_service.sync_user.assert_called_once_with(first_name=first_name, last_name=last_name, email=email)
    # This one passes because usage is correct - keyword arguments are used when calling

def test_user_data_synced_to_crm_create_autospec():
    crm_service = create_autospec(MyCRMService)
    create_user = CreateUser(crm_service=crm_service)
    first_name = "John"
    last_name = "Doe"
    email = "john@doe.com"

    create_user.execute(first_name=first_name, last_name=last_name, email=email)

    crm_service.sync_user.assert_called_once_with(first_name=first_name, last_name=last_name, email=email)
    # This one passes because usage is correct - keyword arguments are used when calling

def test_user_data_synced_to_crm_wrong_usage_magic_mock():
    crm_service = MagicMock()
    create_user = CreateUserWrongUsage(crm_service=crm_service)
    first_name = "John"
    last_name = "Doe"
    email = "john@doe.com"

    create_user.execute(first_name=first_name, last_name=last_name, email=email)

    crm_service.sync_user.assert_called_once_with(first_name, last_name, email)
    # This passes because MagicMock calls whatever you provide, but it should fail because usage is wrong - positional arguments are used when calling instead of keyword arguments

def test_user_data_synced_to_crm_wrong_usage_create_autospec():
    crm_service = create_autospec(MyCRMService)
    create_user = CreateUserWrongUsage(crm_service=crm_service)
    first_name = "John"
    last_name = "Doe"
    email = "john@doe.com"

    create_user.execute(first_name=first_name, last_name=last_name, email=email)

    crm_service.sync_user.assert_called_once_with(first_name, last_name, email)
    # This one is failing because usage is wrong - positional arguments are used when calling instead of keyword arguments

Conclusion

Many times small improvements can make a significant difference on how we perceive a certain thing. It’s the same with automated testing. If we encounter the situation where tests pass, but the deployments are broken, we start to lose trust in our test suite. Using create_autospec is another step towards a place where we can trust our test suite more. It won’t solve all the problems, but it can help greatly.

Happy Python testing!