published on

Python Typeguard Benchmarks

I’ve recently started looking into Python types and how to do them efficiently. The Python typing module turns out to be a very powerful tool introduced in 2015 first to version 3.5.0. The basic idea behind the module was to provide tooling engineers with simple ways of helping developers to know if they are using their methods and classes in a proper way by default or not.

def add(a: int, b: int) -> int:
    return a + b


add(1, 2) # Just fine
add('cat', 'dog') # expected type 'int' got 'str' instead

This is awesome! However, there is no run-time check for python types built in, so despite having the nice error in your IDE, you can still do things like this:

>>> def add(a: int, b: int) -> int:
...     return a + b
... 
>>> add(1, 2)
3
>>> add('cat', 'dog')
'catdog'

Runtime type checking

After looking at this, I was wondering if there’s an actual way to detect run time if a type for a method parameter is correct or not. After a short search, I found the amazing typeguard library.

from typeguard import typechecked


@typechecked
def add(a: int, b: int) -> int:
    return a + b


add(1, 2) # just fine
add('cat', 'dog') #  throws exception: TypeError: type of argument "a" must be int; got str instead

That’s it! We have a runtime error to protect our customers from inputs that we don’t expect! Quite useful, however, a question got raised in my mind. How much does this cost us exactly? So I ran a quick benchmark.

Costs of types

I’ve created a very simple benchmark for testing this. Here’re the functions used:

from typeguard import typechecked


def core_function(a: int, b: int, c: int):
    return a**b**c


def function_without_typeguard(a: int, b: int, c: int):
    return core_function(a, b, c)


@typechecked
def function_with_typeguard(a: int, b: int, c: int):
    return core_function(a, b, c)

I’ve run a total of 100 tests each with a 10000 iterations. Here are the results:

with typeguard
	min:	1.0935659408569336
	max:	3.207068920135498
	median:	1.130236029624939
	mean:	1.3082367300987243
without typeguard
	min:	0.6196920871734619
	max:	2.153690814971924
	median:	0.636016845703125
	mean:	0.7321305203437806

It came out that types seem to be quite expensive in real life, showing a 78.69% uplift in execution time for this particular function.

Conclusions

There could be more tests done here, of course, checking what we can do with complex types, like classes and dataclasses, and there’s also potential to check here how the library scales with the number of parameters that need to be checked per function.

Types are very powerful but they are also expensive. I believe that using a library like typeguard could be very much beneficial for you and your company, however, I recommend using these helpers on external interface definitions, to make sure that once the data enters your system, you’re good to go on the types.

That’s all for today. Thanks for reading and until next time!