Debunking myth "I can't properly test my project because it uses 3rd party API"

Posted May 30, 2021

By Sebastian Buczyński 8 min read

Welcome to the second post from the GRASP series. GRASP stands for General Responsibility Assignment Software Principles. It is a great aid for Object-Oriented Design (but not really exclusive for OOP!). It is all about putting responsibilities in code structures such as classes/methods/modules in such a way it "makes sense".

The challenge

A couple of days ago I participated in a certain Slack discussion. A developer complained he finds the quality of the project he works with low. Two main factors are fragile codebase (when a seemingly insignificant change in one part breaks some other stuff) and almost no tests. The more senior devs told the developer that they can't test the app for real because it has integrations with 3rd party APIs.

I cannot be sure whether this is the whole story or actually the most burning problems. However, I can definitely debunk a myth that 3rd party integrations make it impossible to test the logic of one's application.

Testing the application logic != testing end-to-end

Before we jump into any code or modelling techniques, let's first clarify what's possible and what's not. Obviously, we often won't be able to automate end-to-end tests that are using the production version of a 3rd party service. In order to do any testing, one has to have the capability to:

set a system in the desired state
guarantee isolation between tests
make calls to the system
make assertions about the system's state or responses
(optionally) reset system's state

External systems are not controlled by us and can be unpredictable at times. The only way to get them in the desired state would be to issue calls just like our application does in production. It can have consequences that require a manual intervention to undo.

For the purpose of this article, let's consider a flight reservation system.

# Get a list of available flights from a given airport to a given airport, on a given date for two adults
requests.get(".../api/flights?from=JFK&to=LCJ&adults=2date_start=2021-07-01")

# create a booking for a chosen flight etc
requests.post(".../api/booking", json={...})

We could use the production 3rd party service to find a flight, then book it. But even the first step is not as simple as it looks like. Flights change over time, airports get closed etc. Such tests are not deterministic and over time give headache due to maintenance burden. To conclude, 3rd party integration contributes to instability and makes the application more unpredictable.

If we cannot test the application end-to-end, what's left? Testing the application itself!

The solution - Protected Variations to the rescue!

GRASP's response to a problem of instability is to create a stable interface around them. This principle is called Protected Variations. Don't get distracted by the word "interface". It doesn't necessarily mean one has to use abstract base classes or create an actual interface if we use another programming language.

Protected Variations only suggest one should write a wrapper around flight service API. It can be a class, a function (if it is just a single entry point) or a module with functions inside. That wrapper or I should rather say interface, has to be stable.

Stable interface

Stability doesn't stand for immutable code. It rather means that one should do whatever they can to formalize the contract. That involves:

using type hints
passing DTOs (e.g. dataclasses) as arguments,
getting DTOs out
use dedicated types (e.g. enums) to be more expressive,
avoid leaking information or nomenclature from the API that is not used throughout the project,

Type hints and DTOs are here to enable static code analysis (e.g. mypy). See my other posts about mypy and how it can help. Regarding the implementation of DTOs in Python, here's an excellent article on the subject: 7 ways to implement DTOs in Python and what to keep in mind

@dataclass(frozen=True)
class OneWayFlightsSearch:
    origin: AirportIataCode
    destination: AirportIataCode
    adults: int
    preferred_start: datetime


@dataclass(frozen=True)
class FlightSearchResult:
    booking_token: BookingToken
    total_price: Money
    number_of_stops: int
    legs: List[Leg]

Regarding dedicated data types, we can get more expressive (and type-safe!). You can see a few examples in a snippet above - we have a type for price (Money) or airports - AirportIataCode. Theoretically, we could also create a custom type for the number of adults or stops (only positive integers are valid, not just any!).

Our interface accepts and returns these:

class FlightsApi:
    def search_one_way(
        self, criteria: OneWayFlightsSearch
    ) -> List[FlightSearchResult]:
        ...
        response = requests.get("...")
        ...
        return results  # list of FlightSearchResult

You get the picture of formalizing the contract. One may argue it's a lot of work and additional code. Well, yes it is - but we are not playing code golf. We are taming the instability with appropriate countermeasures.

Stable interface - benefits

What that stability gives us? Surprisingly plenty.

we get a stable point to mock/patch in tests (so we can write acceptance tests of the application! 🎉),
we avoid leaking unnecessary details or information we don't need from a 3rd party,
we can focus on modelling application logic.

Note the most important thing here - it's the application logic that should dictate how input/output DTOs look like and what data we return. Not the 3rd party. If we get back raw strings and our application uses dedicated data types, convert it. If some piece of data is returned from the API but we don't need it in the application - don't pack it into DTO.

Other known incarnations of Protected Variations

Adapter Design Pattern

The interface could be seen as an example of Adapter design pattern as described in Design Patterns: Elements of Reusable Object-Oriented Software book. We could move towards referential implementation by introducing an abstract base class. Even without it, the responsibility of our interface is the same - we adapt 3rd party service to the application.

Hexagonal Architecture (AKA Ports & Adapters) or the Clean Architecture

Hexagonal Architecture is an architectural pattern that fosters separating 3rd party inputs and outputs (ports and adapters from the alternative name). A similar approach is promoted by Uncle Bob's Clean Architecture. It just uses a different name - it is respectively called interface and interface adapter. Umm, did I mention naming building blocks in IT sucks?

Nonetheless, for more information about the Clean Architecture see my first blog post on the subject and 2021 refreshed version.

RDD Stereotypes - Interfacer

Our interface also matches a definition of Interfacer - OOP Stereotype, as understood by Responsiblity-Driven Design:

Interfacer - transforms information and requests between distinct parts of our system
Object Design: Roles, Responsibilities and Collaborations by Rebecca Wirfs-Brock

Testing strategy

Acceptance tests

Now that we have wrapped a source of instability in our application we can write all acceptance scenarios we need. In these tests, we are going to use test-doubles instead of our interface. You can find an introduction to test double types in this article.

Thanks to DTOs and other means of formalizing the contract, we can replace the FlightsApi in tests in a much more reliable way.

Example (simple) scenarios:

when FlightsApi returns no results, we display an error message
when FlightsApi booking fails we give money back.

Interface tests

While the application logic can (and should) be tested using more blocks, the wrapper will be tested mostly in isolation from the rest of the system. We would rather focus on checking if the interface obeys the contract (e.g. returns expected DTOs, raises proper exceptions classes in special conditions etc).

To help ourselves with testing, we may use responses or VCR.py libraries - they will help us write tests against actual responses from the API that we'll freeze and reuse. As a result, these tests will be deterministic.

What if 3rd party service changes API?

If your company is merely one of many clients and the API has been published for a long time then the likelihood of a breaking change is rather low. I would not worry too much about that.

However, if the API is e.g. provided by another team in your company, you may look into contract testing - to make sure your applications understand each other.

What would you say for OOP/Design course in Python?

Object-Oriented Design, GRASP, RDD, Design Patterns, Tactical DDD etc. - there are many guidelines on how to design the code and structure projects. Unfortunately, there are not many good resources on that topic for Python. Not to mention a comprehensive learning resource that combines all of them. Blog posts won't cut it hence I thought of creating a course on the subject. Will that be interesting to you?

If you would like to know when I finish the course, sign up. You won't get any message from me other than relevant to the course - i.e. when it is released + a discount code for trusting me before even seeing the table of contents.

Summary

This was a second article about GRASP which introduced a Protected Variations. The latter is a crucial design principle that helps manage sources of instability and variability in the projects by wrapping them with a stable interface.

It may take a moment to figure out where to exactly draw the line, but it is an investment with great returns. It will work great for projects where 3rd party APIs are used as collaborators for our application. Typical examples are payment providers, some booking services and similar.

Our goal in this design exercise was not to produce something that looks great in a blog post (blogware?!) but rather a practical design that keeps instability at bay. Eventually, as developers, we are to deliver solutions and make sure we can keep doing it over time.