How to patch in Python?

What is (monkey-)patching in Python?

(monkey-) patching is a technique for changing code behaviour without altering its source. It is done in runtime, usually by overriding attributes of existing objects. An object can be an instance of some sort, a class or even a module. The technique is most commonly (ab)used for tests when we cannot pass mocks in a simple way.

Another impressive example is gevent library that turns synchronous code into asynchronous by using monkey-patching.

Let’s benchmark it using wrk:

Gevent made requests a coroutine-friendly library and thanks to concurrency, it enabled our example server to handle over 13.5 times more requests per second.

In the end, we have a program that has coroutine-based concurrency (same principle as in asyncio or node.js) but its code still looks like synchronous one. We do not need special, cooperative libraries or async/await keywords in our code. It’s almost like magic.

Patch in tests

Python includes a utility for patching, i.e. unittest.mock.patch. The default way of using it is to decorate our test function. Assume we have a Django view that looks like this…

and we would like to test it. We notice it has a dependency – ApiClient from another module. If we want to test get_stats view in a predictable, reliable way we need to use a test-double instead of ApiClient. However, there is no simple way to do so. If it was passed to get_stats as an argument, we could simply pass Mock instead.

…but that’s not the case. We can still use patch decorator, though!

This is not end yet, but if we put a debugger in the test, we notice that ApiClient.get_stats_for is now a MagicMock:

It means that our mocking was successful. We replaced a problematic dependency with a Mock. By the way, if you look for best practices for using mocks, check out my (almost) definitive guide about mocking in Python or why mocking can be dangerous when overused.

Now, the test still fails because get_stats receives a MagicMock while it expects a dictionary. We need to parameterize the mock. We can do so by passing a second argument to @patch:

This basically means that instead of api.client.ApiClient.get_stats_for we want a Mock that when called, will return {‘time_to_respond’: timedelta(minutes=1, seconds=3)}.

Patch without decorator

patch can be also used as a context manger. A return result will be a Mock being inserted in a place of an attribute being patched:

“Python patch doesn’t work!” – how to do it right?

Sometimes you will face the situation when despite the presence of patch decorator or context manager, the dependency will look as if it wasn’t patched at all. In short, it may be because there are multiple existing references to the thing you’re trying to patch. The code under test uses one, but you successfully patched another. The operation was successful, but the patient died. What to do?

In short, you need to make sure you patch the same reference that code under test uses.

See Where to patch section of unittest.mock documentation for more details. Alternatively, you can use a nifty alternative to patch, that is patch.object.

patch.object – simpler to get it right

patch.object is dead simple to use – you just import the object whose attribute you want to patch and apply patch.object:

If you want to use patch.object for a method, you import a class. If you want to patch.object a function or entire class, import the module they live in.

Should you patch?

(monkey-) patching should be used sparingly. That ought to be your last resort. In my code, I have no other option but patch thanks to dependency injection.

In the long term price for such tricks is very, very high. Patching often means touching and changing implementation details in a way that was not foreseen by the authors. This introduces extra coupling with things that shouldn’t have it. It means they will be harder to change.

If you really have to, patch only public API of another library or a module in your code.

Image source.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.