mypy: how to use it in my project?

Posted Mar 20, 2020

By Sebastian Buczyński 9 min read

Type annotations are like comments

Type annotations are a great addition to Python. Thanks to them, finally our IDEs are able to provide good quality autocompletion. They did not turn Python into statically typed language, though. If you put a wrong annotation (or forget to update it after code change), Python will still happily try to execute your program. It just may fail miserably. Type annotations are like comments - they do not really have any influence on the way how your program works. They have also the same disadvantage - once they become obsolete, they start leading developers astray. Type annotations advantage is that they have a very specific format (unlike comments) so can be used to build tools that will make your life easier and code better. In this article, you will learn how to start using mypy, even if you like to add it to your existing project.

For the needs of this article, I will be using a large legacy monolithic project written in Django. I won't show any actual snippets from it (because I have NDA on that) but I will demonstrate how mypy can be added even to such a messy codebase.

Step 1: install mypy

The first step is as easy as pip install mypy

mypy works like a linter - it performs static code analysis just like pylint or pycodestyle. Hence, if you split your dependencies into dev and "production" ones, you want to include mypy in the first category. If you use poetry, you could do with the command: poetry add --dev mypy

Don't run it yet because you will only see dozens to thousands of errors you can't really do anything about.

Step 2: Create the most basic mypy.ini

Create a config file in the root directory of your backend code and call it mypy.ini:

[mypy]
ignore_missing_imports = True

If you decided to ignore my warning and run mypy, it must have complained a lot about Skipping analyzing '': found module but no type hints or library stubs

That happens because mypy tries to check if 3rd party libraries. However, a lot of Python libraries is simply not type-annotated (yet), so that's why we ignore this type of error. In my case (legacy Django project) this particular error was raised 3718 times.

Ideally, you would now see zero complaints but that's rarely a case. Even though I have no type annotations, mypy is still able to find some issues thanks to inferring (guessing) types. For example, it is able to tell if we use a non-existent field of an object.

Dynamic typing versus mypy

Before showing the next step, let's digress for a moment. Even though mypy complains about a few code lines it does not necessarily mean the code there won't work. Most likely it does, it is just that mypy is not able to confirm that. When you start to write type annotations you will learn to write code in a bit different way. It will be simpler to analyse by mypy so it will complain considerably less.

Bear in mind that mypy is still in active development. At the moment of writing this article, it was in version 0.770. The tool may sometimes give false negatives i.e. complain about working code. In such a case, when you are certain the code works (e.g. is covered by tests) then you just put # type: ignore comment at the end of the problematic line of code.

Step 3: Choose one area of code you want to type-annotate

It is unrealistic to expect that introducing type annotations and mypy is possible in a blink of an eye. In legacy projects (like the one I experiment on) it would be a titanic effort to type-annotate all the code. Worse, it could bring no real benefit because certainly some areas are not changed anymore. I am sure there is a plenty dead code. Moreover, mypy will definitely affect the way of working on the project for the whole team. Lastly, we may simply come to the conclusion it is not working for us and want to get rid of that.

The point I am trying to make is - start small. Choose one area of code and start adopting mypy there. Let's call this an experiment.

My legacy Django project consists of 28 applications. I could just choose one of them but I can go even further, for example, enforce type hints in just one file. Go with the size you are comfortable with. As a rule of thumb, you should be able to type-annotate it in less than 2 days, possibly less.

I've chosen an area that is still used but not changing too often except for rare bug fixes. Let's say the application I will type-annotate is called "blog".

Step 4: Turn off type checking in all areas except your experiment

Now, change your mypy.ini to something like:

[mypy]
ignore_missing_imports = True
ignore_errors = True

[mypy-blog.*]
ignore_errors = False

Where blog is a module you want to start with. If you would like to start with an even narrower scope, you can add more submodules after the dot.

[mypy]
ignore_missing_imports = True
ignore_errors = True

[mypy-blog.admin.*]
ignore_errors = False

Step 5: Run mypy

Now, type mypy . This will once again print errors, but hopefully not too many. In my case, it is just 9 errors in 3 files. Not that bad.

Step 6: Fix errors

As I mentioned above, there are certain patterns I would say that make mypy freakout. As an exercise, you should rewrite the code or just learn how to put # type: ignore comment :)

In my code, 4 out of 9 errors concerned dead code, so I removed it.

Another one was complaining about Django migration. Since I have no interest in annotating it, I disabled checking migrations path in mypy.ini.

[mypy]
ignore_missing_imports = True
ignore_errors = True

[mypy-blog.*]
ignore_errors = False

[mypy-blog.migrations.*]
ignore_errors = True

Remaining four errors were all found in admin.py file. One of them complained about assigning short_description to a function:

# mypy output
error: "Callable[[Any, Any, Any], Any]" has no attribute "short_description"

# in code
def delete_selected(modeladmin, request, queryset):
    ...

delete_selected.short_description = "Delete selected SOMETHING #NDA"

mypy is right by saying the function indeed does not have short_description. On the other hand, this is Python and functions are objects. Hence, we can dynamically add properties to it in runtime. Since this is Django functionality, we can safely ignore it.

delete_selected.short_description = "Delete selected article language statuses"  # type: ignore

Three errors left. All of them are the same and they are false negatives (but blame's on me, I fooled mypy into thinking the code will not work)

# mypy output
error: Incompatible types in assignment (expression has type "Type[BlogImage]", base class "MyAdminMixin" defined the type as "None")

# in code
class BlogImageInline(MyAdminMixin, admin.TabularInline):
    model = BlogImage  # this line is problematic

class MyAdminMixin:
    model = None

Simply saying, we inherit from a class that has a field declared with default value None. It is always overridden in subclasses, but mypy thinks we are doing something nasty that way. Well, in reality, we're gonna always use a subclass of Django model here, so let's just type annotate our mixin and get rid of final 3 errors:

from django.db import models
from typing import Type


class MyAdminMixin:
    model: Type[models.Model]

Step 7: Turn on more restrictive checks

By default mypy only checks code that either has type annotations or the type can be inferred. It doesn't force writing type annotations on you, though eventually, you want it. It is much simpler to enforce it when starting a greenfield project, but not impossible in legacy codebases.

There is a lot of options to find out, but let's start from the most useful two:

disallow_untyped_defs
disallow_untyped_calls

Just put them in your mypy.ini with value = True to start getting errors for missing type annotations.

[mypy]
ignore_missing_imports = True
ignore_errors = True

[mypy-blog.*]
ignore_errors = False
disallow_untyped_defs = True
disallow_untyped_calls = True

There are plenty of other options worth checking out. See The mypy configuration file.

Step 8: Fix errors

Now I got 122 errors from 13 files. The most common one is complaint about missing type annotations in a function. In other words, mypy wants us to put annotations for arguments and return types.

error: Function is missing a return type annotation

It doesn't mean I have to do all this mundane work at once. For example, 62 out of 122 are coming from tests. I can as well disable checks there (at least temporarily) to focus on annotating views, serializers and models.

[mypy]
ignore_missing_imports = True
ignore_errors = True

[mypy-blog.*]
ignore_errors = False
disallow_untyped_defs = True
disallow_untyped_calls = True

[mypy-blog.tests.*]
disallow_untyped_defs = False
disallow_untyped_calls = False

Then, start adding annotations to functions...

# before
def translate_blog_post(source_language_id, destination_langauge_id):
    pass

# after
def translate_blog_post(source_language_id: int, destination_langauge_id: int) -> None:
    pass

Let mypy guide you. Run it often to fix new issues as they appear. For example, when you type annotate your functions, it will tell you about places when you misuse them, for example by giving None as an argument even though type annotation specifies it should be int.

The whole process can be tiresome, but it is the best way to learn how to write type annotations.

Step 9: Repeat steps 7-8

That's it for our short guide of adopting mypy :) Stay tuned for the next two parts where we are going to explore ways to automate type annotating codebases and learn about awesome tools that use type annotations to make your life easier and code better.

What can't be covered by this is series is how to efficiently write type annotations. You need to practice on your own. Don't forget to check out mypy and typing module's documentation (links below).

Let me know in comments if you have encountered any interesting problems when trying to adopt mypy!