Prefer Consumer over Integration Tests

Recently I attended an RCA meeting for an incident that happened in prod few weeks back. The issue was somewhat of a 2nd degree failure. To put it in simple terms, lets say a workflow composed of two services - A and B. When user performed an action from the UI, it made a call to Service A which then internally called service B with some parameters based on user preferences. In this case, due to some unusual combination of preferences, Service-A ended making call to Service-B with parameters that Service-B didn’t fully understood.

Solution - Write an integration test to cover this scenario.
Reasoning - Since this involved 3 different components (User Preferences, Service A and Service B), it should be covered by an integration test.

More Reasoning: This is sort of the usual way of thinking about the integration tests - meaning whenever there are multiple services involved, we immediately think about writing an integration test. But is this the right approach specially in microservice architecture? Lets try to give it some more reasoning. In microservice architecture, we try to create hundreds of smaller services that have identity of their own within a bounded context. In which case for any given workflow, there will always be multiple services interacting with each other. Now if we follow the reasoning above, we will end up writing integration tests for every possible interaction. The testing pyramid would change: 


Why testing pyramid is so important? It signifies the importance of reliable and fast testing strategies. We want to write fast and stable tests! Isolated tests, which could be Unit or Component tests, are fast and stable, so they should carry the heavy weightage. Integration/E2E tests are slow and unstable and significantly impacts our CD pipelines and our ability to deliver code to production. If you ever find yourself in a situation where testability of your product relies more on the integration tests - you have Integration Test Hell.


Consumer Tests to rescue: Consumer tests is an interesting idea where we make Consumer services responsible for making sure the Producer of an API does not break their use case. If a use case is broken, its Consumer’s responsibility because they didn’t notify the Producers about its use case. It is achieved by allowing Consumers to submit their test cases into the Producer’s Isolated testing pipeline. In the scenario above, Service A is a consumer of an API owned by Service B. If Service-A required Service-B to perform some action under some particular scenario, it should submit its use case into Service-B’s pipeline. Wait.. does it mean now I need to write 3 isolated test cases instead of 1 integration test? YES!


Advantages of Consumer tests:
  • Shared responsibility - Consumer is responsible for making sure it notifies about its use cases to the Producer and which in turn should make sure those use cases are never broken.
  • Creates a platform for Contract based development. 
  • Reduces the load on integration tests and therefore faster component deployments. 
  • Producers have a very good understanding on how its services are used. 




Popular posts from this blog

Break functional and orchestration responsibilities for better testability

Image
Microservice architecture paradigm fits well within the Unix’s first principle/philosophy: “Make one program do one thing well”.  We love creating small services which has identity of its own (within a bounded context). These services have their own CI/CD pipelines and provides loose coupling b/w components which significantly improves our ability to deliver code to production. In reality, a workflow can span multiple services. Its very common to find scenarios like below: In the above example, a workflow spans 3 services (Service-1, 2 and 3). Service1 receives the original request. In order to process this request, it GET(s) some data from its dependent services(d1/d2/d3). It further runs some business logic and produces an intermediate response #A. It then makes a state mutable call (POST/PUT/DELETE/PATCH) to Service-B using #A as input and waits for its response. Service-B follows the same paradigm and the workflow ends when all services returns.  The scena...
Read more

Microservices and Tech Stack - Lessons Learned

Image
In this blog, I want to talk about how our technology stack has evolved over time while transitioning from Enterprise to adopting Microservice architecture as a main stream development practice and share some of the lessons learned. Tech stack for us has evolved from being a Monolithic Enterprise to Cloud to now Microservices. During Enterprise days, it looked something liked this: At the app layer, we had a desktop client written in Windows form and WPF. The app would talk to the Service Layer which was SOA architecture written completely in C#. It was the only language allowed for us to use. These were monolithic and stateful services communicating to each other over WCF. We used SQL server as backend storage. All this get deployed on premise meaning our clients buy our software and host it on their own hardware. I work for Finance industry (Stock Market to be precise) and they were sort of late adopter of public cloud. They absolutely hated the idea of putting their data on p...
Read more