Stop naming your python modules “utils”

Imagine the following situation: there is a software developer that is either adding new code or refactoring existing one by extracting a class/function. They have to place code somewhere, but it does not seem to fit anywhere. So what does a developer do? They create a new module – utils.py.

Why utils is a terrible name?

utils is arguably one of the worst names for a module because it is very blurry and imprecise. Such a name does not say what is the purpose of code inside. On the contrary, a utils module can as well contain almost anything. By naming a module utils, a software developer lays down perfect conditions for an incohesive code blob. Since the module name does not hint team members if something fits there or not, it is likely that unrelated code will eventually appear there, as more utils. More on it later.

Synonyms of utils, like helpers, commons, etc. are bad for the same reason.

Why people do this?

Excuse I – it is just one function

Initially, yes – it may be just one function. One function in a badly named module is not that wrong, isn’t it?

It is. Similarly to the broken windows theory, one occurrence of misbehaviour invites more of them. One function or class in utils is a small problem, indeed. Hence, it should be refactored when it is easy. Once the utils module grows, it will require a lot more effort to split it. And surprise, surprise, no one will be willing to do that.

How bad it can get? Once, in one Python repository, I saw there were several utils.py modules. My favourite contained 13 various functions and one utility class. What were these functions doing? Everything, from validation to data normalization to saving to the database to sending HTTP requests to getting current datetime formatted accordingly to the format parameter (Yes, separate, loose functions).

That’s how programming hell looks like. utils.py quickly becomes a whirlpool for all code that does not fit other places. It smoothly leads us to excuse number 2…

Excuse II – There is no other place to put this code

Indeed, there may be no place that a new class/function fits. Reaction to create a new place for the code is good. However, a programmer needs to put more effort when thinking about the name. As we know, taking the easiest road with utils is a slippery slope.

We can do better by naming module by the purpose of functions living inside. If they will be creating other objects, let’s name it factories.py. If their role is validation – go for validators.py. Maybe we need a few functions that operate on phone numbers? See if they could not be a regular, stateful class PhoneNumber and just put it in a separate file – phone_number.py.

A special case – functions with business logic. There are many techniques for that, some of them are more sophisticated than others (e.g. the Clean Architecture) but let’s consider a simple case. Assuming we have Django + DRF web application that contains business logic in serializers. We slowly migrate our API to version 2 and we need to put business logic extracted from V1 serializer in some other place, so that serializer V2 may reuse that. DO NOT PUT THIS IN utils.py. Try putting business logic in services.py module. Name service comes from an application service – a single thing that the application does for the clients. If this was, for example, booking a flight, a service could be named flight_booking_service and would:

  • authorize payment on customer’s payment card
  • reserve a flight using 3rd party provider
  • send an email (or scheduled a Celery task to do so)

Excuse III I need a place for company commons

Let’s say you are building a distributed application and there are chunks of code that needs to be reused in a majority or all microservices. It is a natural reaction to put them together in someplace, like a separate repository to be installed as a package. But please don’t call it {company_name}-utils. I heard about a case of such a repository, but luckily for its maintainers, it was not that big. It contains code responsible for:

  • secrets handling, using public cloud services
  • logging configuration that uses specific public services

As I said, it’s not that bad but it would be nice if they were more specific with the name, for example, cloud-toolkit or split that into two separate repositories/packages because frankly there are microservices that use only one of functionalities.

Excuse III – But Django does that

Yes, there is a couple of utils packages in Django. Shame on them for using utils name. However, notice that at least some of them could be separated from the framework and bundled as optional dependencies. Also, at least they are grouped in cohesive sub-packages – e.g. django.utils.timezone or django.utils.translation.

Unless you are writing a framework, stay away from utils. 😉

Are all utils bad?

Not exactly. Eventually, one may need a couple of auxiliary functions. In that case, organize such code in modules named by theme – like datetimes, phone_numbers, etc. Such functions should be pure (in terms of functional programming).

Pure Functions – do not have side effects, that is, they do not change the state of the program. Given the same input, a pure function will always produce the same output.

https://stackabuse.com/functional-programming-in-python/

Summary

Do not use utils as a name for your Python module neither put it into a class name. Try to be more specific about the role of code – e.g. create a place for validators, services or factories. If the role criterion doesn’t help and you really dealing with utils, try grouping code by its theme –

utils modules are dangerous, because they deteriorate over time. Each and another person that adds something that does not fit anywhere will happily add it to the utils module, increasing its incohesion. The disorder will grow over time, becoming greater and greater burden to work with.

If you see a newly created utils module in a code review, request it to be renamed. If you are tempted to add something to existing utils, create a new place for your code and move there everything from utils that fits a newly created module.

In the end, you will exercise your brain to become better at designing code.

Further reading:

Image source

30

8 thoughts to “Stop naming your python modules “utils””

  1. A commit from this morning:

    commit 3a87c6c63e713744748eed76510000d5298baa69
    Author: username
    Date: Tue Apr 14 09:47:18 2020 -0500
    Broke foo.utils into foo.timestamp and foo.services
    “I read on the Internets” that naming a module ‘utils’ was just
    being lazy. Moved the timestamped_path function in foo.timestamp
    and moved the service_a_csv and upload_to_service_b functions to
    foo.services.

    1. Hi Erik, thanks for the comment.
      I wouldn’t call the reason for naming a module utils laziness. It’s rather caused by no prior bad experience with legacy code suffering from it or lack of a technical mentorship. Laziness argument could be used to put a blame on someone unaware of bad consequences and that’s just not right. Let’s hope people learn from the mistakes of others, including mine – that’s why I wrote this blog post. 😉

      1. In my particular case, it was indeed laziness that caused me to dump unrelated functions into a module named ‘utils’. I apologize for the mischaracterization of your arguments (which were convincing). I wanted you to know that your post reached at least one person and caused a change in their (my) behavior.

  2. Nice post, something we’ve all been through. However, this post might’ve been better with some concrete illustrations of how utils can be better named. (Came here as it was featured in pycoder’s weekly).

    1. Thanks for the comment and a valuable suggestion; I agree with the statement that more practical examples that those from the article (group by functionality or by theme) would help, but they deserve a more in-depth analysis. It would be hard to give a bit of general advice but why not a more detailed case study? Perhaps a topic for another blog post, though 😉

  3. I don’t fully agree, there is a reason for a utils.py module to exists. It’s the place for functionality that helps you writing code. Your project has some purpose and most functionality you add to it is to solve a project specific problem. However sometimes you need to include very general functionality, not at all project specific. For example my utils.py contains a function for removing duplicates from a list of non-hashable objects as well as Python 3.8’s implementation for singledispatchmethod since my project uses Python 3.7. The content of utils.py should be minimal and it must not depend on any other project internal files. Rather one should expect it to be imported into any other module of the project. More importantly, the contained functions must not solve project specific tasks, but rather have general functionality. Like a toolbox but too small to be bundled as a separate package.

    1. Thanks for the comment. I agree that a minimal utils.py should not be considered a problem – no matter how you measure it, numbers won’t come out alarming. My point is that even a small module with such imprecise name deteriorates over time. Unless you have total control over contents (maybe you are the only developer, maybe there are not many changes and you can always do the code review or have other means of control) then 100% fine. In other cases, having utils is asking for troubles

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.