When to use metaclasses in Python: 5 interesting use cases

Metaclasses are mentioned among the most advances features of Python. Knowing how to write one is perceived like having a Python black belt. But are they useful at all outside job interviews or conference talks? Let’s find out! This article will show you 5 practical applications of metaclasses.

The legend says there was a Python developer that actually used metaclasses in their code

What metaclasses are – quick recap

Assuming you know the difference between classes and objects, metaclasses should not be that difficult – they are classes for classes (hence “meta” in their name).

Simply saying – while classes are blueprints for objects, metaclasses are blueprints for classes. Class acts as a blueprint when we create an instance of it whereas metaclass acts as a blueprint only when a class is defined.

The simplest implementation of a metaclass that does nothing looks as follows:

Avoiding decorators repetition or decorating all subclasses

Let’s say you fall in love with relatively recent dataclasses stdlib module or you use much more advanced attrs. Or you just use a lot of repetitive decorators on your classes:

Maybe it’s not too much repetition, but we can still get rid of it we we wrote a metaclass for Event:

The most important line, attr.s(frozen=True, auto_attribs=True)(new_cls) deals with decoration of our subclasses. The key to understanding this example is the fact that syntax with “at sign” (@) is merely a syntax sugar over such construct:

Validation of subclasses

Speaking of classes, let’s think about inheritance in a context of Template Method design pattern. Simply saying, we define an algorithm in the base class, but we leave one or more steps (or attributes) as abstract methods (or properties), to be overridden in a subclass. Consider this example:

We have a simple base class for JSON exporters. It is enough to subclass it and provide an implementation for the _filename property and the _build_row_from_raw_data method. However, abc only provides validation for the absence of these parameters. If we, for example, would like to check more things, like the uniqueness of filenames or its correctness (e.g. always ends with .json) we can write a metaclass for that as well:

Speaking of ABC, this is another thing that’s implemented using metaclasses. See this talk by Leonardo Giordani – Abstract Base Classes: a smart use of metaclasses.

Registering subclasses – extendable strategy pattern

Using metaclasses attributes we can also write a clever, Open-Closed implementation of a factory. The idea will be based on keeping a registry of concrete (non-abstract) subclasses and building them using a name:

Warning: for this to work, we need to import all of the subclasses. If they were not loaded to the memory of interpreter, they simply will not be registered.

A less abstract example could be a plugin system for a linter (think about Pylint or flake8). By subclassing abstract Plugin class, we would be not only providing our custom checks but also registering the plugin. By the way, if you are looking how to write such a plugin for Pylint – check out my article on Writing custom checkers for Pylint.

A declarative way of building GUI

Credit for this way amazing metaclasses application goes to Anders Hammarquist – author of EuroPython talk Metaclasses for fun and profit: Making a declarative GUI implementation.

Basically, the idea is to turn imperative code responsible for building a GUI out of components…

…into this:

Just wow. It’s impressive, because not only simplifies the resultant code, but also fits our minds much better. Visual composition by nesting classes looks more natural to us, given that the end result is very similar – components nested within each other.

If you are interested about implementation details (and problems author had to overcome) see the talk. Code is available on bitbucket: https://bitbucket.org/iko/ep2016-declarative-gui/

Adding attributes – Django ORM’s Model.DoesNotExist

If you have some experience with Django, you surely noticed that each model class gets dedicated DoesNotExist exception. The latter is an attribute on the class. But where does it come from? Well, Django uses metaclasses. For models, it does a lot of things from validation to dynamically adding a few attributes, i.e. DoesNotExist and MultipleObjectsReturned exceptions.

Honourable mention: __init_subclass__

As you have noticed, metaclasses are quite verbose. For example, if we want to affect the entire hierarchy of classes, we need at least two classes (one for the metaclass and another one for the base class).

There is also a risk of falling into metaclasses conflict if we try to apply it in the middle of a hierarchy. For example, you won’t be able to just use a custom metaclass for your Django model. You would have to use the trick I was leveraging – create a subclass of django.db.models.Model (django.db.models.base.ModelBase), write your own logic in __new__ and then create your own base class for all models, instead of using django.db.models.Model. Sounds like a lot of work.

Luckily for us, since Python3.6 there is another hook available: __init_subclass__. It is able to replace majority (if not all) metaclasses.

For details, see PEP 487.

Summary

I know what you are thinking – all these metaclasses are useful only if you are writing a framework. That would be correct, but up to the point. If you discover a repetitive pattern in your project, why not think about metaclasses?

The “meta” part is the key to see their usefulness 😉 When you see that you’re spending a lot of time or making mistakes while doing “the work”, maybe there’s a room for some “metawork”!

Metaclasses can be used to the enforcement of conventions, guiding implementation or enable easy extendability.

Do you know any other cool application of metaclasses? Share them in comments!

Source of image: https://pixabay.com/pl/illustrations/excalibur-miecz-polanie-kaprys-3445952/

1

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.