Type annotations are like comments
Type annotations are a great addition to Python. Thanks to them, finally our IDEs are able to provide good quality autocompletion. They did not turn Python into statically typed language, though. If you put a wrong annotation (or forget to update it after code change), Python will still happily try to execute your program. It just may fail miserably. Type annotations are like comments – they do not really have any influence on the way how your program works. They have also the same disadvantage – once they become obsolete, they start leading developers astray. Type annotations advantage is that they have a very specific format (unlike comments) so can be used to build tools that will make your life easier and code better. In this article, you will learn how to start using mypy, even if you like to add it to your existing project.
For the needs of this article, I will be using a large legacy monolithic project written in Django. I won’t show any actual snippets from it (because I have NDA on that) but I will demonstrate how mypy can be added even to such a messy codebase.
Step 1: install mypy
The first step is as easy as
pip install mypy
mypy works like a linter – it performs static code analysis just like pylint or pycodestyle. Hence, if you split your dependencies into dev and “production” ones, you want to include mypy in the first category. If you use poetry, you could do with the command:
poetry add --dev mypy
Don’t run it yet because you will only see dozens to thousands of errors you can’t really do anything about.
Step 2: Create the most basic mypy.ini
Create a config file in the root directory of your backend code and call it mypy.ini:
[mypy] ignore_missing_imports = True
If you decided to ignore my warning and run mypy, it must have complained a lot about
Skipping analyzing '
That happens because mypy tries to check if 3rd party libraries. However, a lot of Python libraries is simply not type-annotated (yet), so that’s why we ignore this type of error. In my case (legacy Django project) this particular error was raised 3718 times.
Ideally, you would now see zero complaints but that’s rarely a case. Even though I have no type annotations, mypy is still able to find some issues thanks to inferring (guessing) types. For example, it is able to tell if we use a non-existent field of an object.
Dynamic typing versus mypy
Before showing the next step, let’s digress for a moment. Even though mypy complains about a few code lines it does not necessarily mean the code there won’t work. Most likely it does, it is just that mypy is not able to confirm that. When you start to write type annotations you will learn to write code in a bit different way. It will be simpler to analyse by mypy so it will complain considerably less.
Bear in mind that mypy is still in active development. At the moment of writing this article, it was in version 0.770. The tool may sometimes give false negatives i.e. complain about working code. In such a case, when you are certain the code works (e.g. is covered by tests) then you just put
# type: ignore comment at the end of the problematic line of code.
Step 3: Choose one area of code you want to type-annotate
It is unrealistic to expect that introducing type annotations and mypy is possible in a blink of an eye. In legacy projects (like the one I experiment on) it would be a titanic effort to type-annotate all the code. Worse, it could bring no real benefit because certainly some areas are not changed anymore. I am sure there is a plenty dead code. Moreover, mypy will definitely affect the way of working on the project for the whole team. Lastly, we may simply come to the conclusion it is not working for us and want to get rid of that.
The point I am trying to make is – start small. Choose one area of code and start adopting mypy there. Let’s call this an experiment.
My legacy Django project consists of 28 applications. I could just choose one of them but I can go even further, for example, enforce type hints in just one file. Go with the size you are comfortable with. As a rule of thumb, you should be able to type-annotate it in less than 2 days, possibly less.
I’ve chosen an area that is still used but not changing too often except for rare bug fixes. Let’s say the application I will type-annotate is called “blog”.
Step 4: Turn off type checking in all areas except your experiment
Now, change your mypy.ini to something like:
[mypy] ignore_missing_imports = True ignore_errors = True [mypy-blog.*] ignore_errors = False
Where blog is a module you want to start with. If you would like to start with an even narrower scope, you can add more submodules after the dot.
[mypy] ignore_missing_imports = True ignore_errors = True [mypy-blog.admin.*] ignore_errors = False
Step 5: Run mypy
mypy . This will once again print errors, but hopefully not too many. In my case, it is just 9 errors in 3 files. Not that bad.
Step 6: Fix errors
As I mentioned above, there are certain patterns I would say that make mypy freakout. As an exercise, you should rewrite the code or just learn how to put
# type: ignore comment 🙂
In my code, 4 out of 9 errors concerned dead code, so I removed it.
Another one was complaining about Django migration. Since I have no interest in annotating it, I disabled checking migrations path in mypy.ini.
[mypy] ignore_missing_imports = True ignore_errors = True [mypy-blog.*] ignore_errors = False [mypy-blog.migrations.*] ignore_errors = True
Remaining four errors were all found in admin.py file. One of them complained about assigning short_description to a function:
# mypy output error: "Callable[[Any, Any, Any], Any]" has no attribute "short_description" # in code def delete_selected(modeladmin, request, queryset): ... delete_selected.short_description = "Delete selected SOMETHING #NDA"
mypy is right by saying the function indeed does not have short_description. On the other hand, this is Python and functions are objects. Hence, we can dynamically add properties to it in runtime. Since this is Django functionality, we can safely ignore it.
delete_selected.short_description = "Delete selected article language statuses" # type: ignore
Three errors left. All of them are the same and they are false negatives (but blame’s on me, I fooled mypy into thinking the code will not work)
# mypy output error: Incompatible types in assignment (expression has type "Type[BlogImage]", base class "MyAdminMixin" defined the type as "None") # in code class BlogImageInline(MyAdminMixin, admin.TabularInline): model = BlogImage # this line is problematic class MyAdminMixin: model = None
Simply saying, we inherit from a class that has a field declared with default value None. It is always overridden in subclasses, but mypy thinks we are doing something nasty that way. Well, in reality, we’re gonna always use a subclass of Django model here, so let’s just type annotate our mixin and get rid of final 3 errors:
from django.db import models from typing import Type class MyAdminMixin: model: Type[models.Model]
Step 7: Turn on more restrictive checks
By default mypy only checks code that either has type annotations or the type can be inferred. It doesn’t force writing type annotations on you, though eventually, you want it. It is much simpler to enforce it when starting a greenfield project, but not impossible in legacy codebases.
There is a lot of options to find out, but let’s start from the most useful two:
Just put them in your mypy.ini with value = True to start getting errors for missing type annotations.
[mypy] ignore_missing_imports = True ignore_errors = True [mypy-blog.*] ignore_errors = False disallow_untyped_defs = True disallow_untyped_calls = True
There are plenty of other options worth checking out. See The mypy configuration file.
Step 8: Fix errors
Now I got 122 errors from 13 files. The most common one is complaint about missing type annotations in a function. In other words, mypy wants us to put annotations for arguments and return types.
error: Function is missing a return type annotation
It doesn’t mean I have to do all this mundane work at once. For example, 62 out of 122 are coming from tests. I can as well disable checks there (at least temporarily) to focus on annotating views, serializers and models.
[mypy] ignore_missing_imports = True ignore_errors = True [mypy-blog.*] ignore_errors = False disallow_untyped_defs = True disallow_untyped_calls = True [mypy-blog.tests.*] disallow_untyped_defs = False disallow_untyped_calls = False
Then, start adding annotations to functions…
# before def translate_blog_post(source_language_id, destination_langauge_id): pass # after def translate_blog_post(source_language_id: int, destination_langauge_id: int) -> None: pass
Let mypy guide you. Run it often to fix new issues as they appear. For example, when you type annotate your functions, it will tell you about places when you misuse them, for example by giving None as an argument even though type annotation specifies it should be int.
The whole process can be tiresome, but it is the best way to learn how to write type annotations.
Step 9: Repeat steps 7-8
That’s it for our short guide of adopting mypy 🙂 Stay tuned for the next two parts where we are going to explore ways to automate type annotating codebases and learn about awesome tools that use type annotations to make your life easier and code better.
What can’t be covered by this is series is how to efficiently write type annotations. You need to practice on your own. Don’t forget to check out mypy and typing module’s documentation (links below).
Let me know in comments if you have encountered any interesting problems when trying to adopt mypy!
- The mypy configuration file
- Type hints cheat sheet
- typing module documentation
- Dropbox: Our way to type checking 4 millions of Python