The goal of tests is to make sure that our software is working as expected. Sometimes, it’s enough to use a single example to prove that things meet our expectations. For example, when testing, we can fetch a user from our store after adding it. On the other hand, we need more examples to verify that the email address validator isn’t marking valid email addresses as invalid for no reason. In such cases, test parametrization comes in handy. This way we can write a single test and apply it to multiple examples to really make sure our code is working as expected.
Parametrize decorator
Fortunately, pytest comes with built-in parametrize decorator which we can use to parametrize our tests. It’s simple to use. We just need to define names of variables and list of examples:
This way, we can easily extend a list of examples for which a particular behavior must be implemented. In this case, we listed all forms of valid emails that we can think of. If someone reports a bug, “I can’t sign up with my email address. The app is saying it’s invalid email, but I’m using it daily for everything else.”, we can add a new example and update our function to work for another case as well.
More examples, more confidence
Imagine having only one test with one valid email example. How would that work? It would be very dangerous to change the function. It would be very easy to make it work only for the new example but break it for some of the cases that worked previously. That’s where parametrization shines, indeed. Since it’s easy to keep previous examples and add new ones, you can always ensure your changes haven’t broken something. This increases the confidence in your test suite.
Do not abuse it
Throughout my career, I’ve seen many people learning about parameterize decorator and suddenly going into “Let’s use it for everything” mode. While it offers great benefits, it’s also a trap. We should follow the same test best practices when we’re using it and we’re not using it.
- A test should assert a single behavior.
- A test must not contain IF statements.
What does that mean when writing tests? Use a single test for a single behavior. For example, use one test for valid email addresses and another for invalid ones as you can see above. Don’t try to write a single test and parametrize the output as well – as you can see in the examples below.
But why not? Because abusing it makes tests hard to read. It’s also harder to see which behavior is broken if only one test is used and some examples fail. With a single behavior per test, we can immediately know what went south – just by the name of the failed test. Even when using parametrization.
Conclusion
As with any other thing, with great power comes great responsibility. Parametrization provided by pytest can help us a lot when we need to cover multiple examples to ensure our software behaves well. It allows us to write less code to achieve the same result – which is great. We just need to be careful not to over-use it.
You can find full code examples in this Github repository.