Microservices

 In today's fast-paced digital landscape, organizations are constantly striving for agility, scalability, and innovation. Microservices architecture has emerged as a powerful paradigm for achieving these goals. However, the successful implementation of microservices is not solely dependent on technical aspects. It is equally vital to ensure organizational alignment, as misalignment can lead to severe challenges. In this article, we will explore the crucial need for organizational alignment in the context of microservices, drawing insights from pioneering companies, and delves into the repercussions of misalignment, supported by examples of both good and bad organizational behavior patterns. Organizational alignment in the context of microservices refers to the harmonious synchronization of various organizational units, including development teams, operations, and business functions, to support the principles and practices of microservices architecture. A Good organizational alignme...

Caching in general is a good practice to improve the responsiveness of the system. But sometimes it can go horribly worng and cause cascading failures leading to bringing the entire system down. Below were such incidents that I encountered: Backend service caches M2M auth tokens which it gets by making a API calls to an Authservice. Auth service introduced a bug in which it returns empty token and 200 status code. The backend service checks the response code and cache the token for few hours and the rest is history :) Service caches DNS entry to make API call for the resource. DR hits and the cached DNS entry no longer works. In the absence of dynamic service discovery, needs entire cluster restart. Caching a /GET call and not respecting the expiration. Well this is just silly and should have been caught in the code review Happy caching!

Companies have a never ending quest to continuously adopt new tools, processes and new architectures with the goal of enabling teams to move fast and build high performing teams. What is a high performing team? If you ask a product manager, they will say "It’s a team that can deliver new value faster". If you ask a Team lead they will say along the lines of "A high performing team can pay down its tech debt faster, spends majority of the time building new value to the product and optimizes their work for improved quality and faster delivery." "so what makes these teams move fast?" Now this is a great question! I have been fortunate to work with some of these high performing teams.  Juggling down the memory lane, I was trying to find the answer to this question in a way that discounts the personal traits of any individual in the team. I stumbled upon three unique qualities that distinguished a high performing team from others: Ability to make sweeping chang...

Terminology Observability : Observability is the property of a system to answer either trivial or complex questions about itself. How easily you can find answers to your questions is a measure of how good the system’s observability is. Monitoring : observing the quality of system performance over a time duration Instrumentation : refers to metrics exposed by the internals of the system using code (a type of white-box monitoring) Why IOM?  Analyzing long-term trends like User growth over time User time in the system System Performance over time Comparing over time or experiment groups How much faster are my queries after new version of a library Cache hit/miss ratio after adding more nodes Is my site slower than it was last week? Alerting - Something is broken, and somebody needs to fix it right now!  Building dashboards - Answer basic questions about your service - Latency, Traffic & Errors. Conducting ad hoc retrospective analysis - Our latency just shot up; what else ha...

Here is the link to my #Dockerccon2020 talk "Monolithic to Microservices + Docker = SDLC on Steroids!" 

EWF is another microservice decomposition pattern that I like to use when building services. It stands for Entity, Workflow & Functional services. In simplest terms, it’s an extension of Single Responsibility principle and allows for building independent testable services. To understand this pattern better, let’s start with an example. Imagine I want to write a signup service that allows user to signup to a website using their Google account. The service would perform the following:  Let user login into google account and fetch user’s basic information from Google upon successful sign-in. Apply more default settings associated to creating new user.  Persist the user in the database.  After successful registration, redirect the user to the homepage.  At a high level, this works just fine and gets the job done. But if you think about testability first and wants to test this service in isolation we have a problem! The entire testing paradigm relies on ...

Recently I attended an RCA meeting for an incident that happened in prod few weeks back. The issue was somewhat of a 2nd degree failure. To put it in simple terms, lets say a workflow composed of two services - A and B. When user performed an action from the UI, it made a call to Service A which then internally called service B with some parameters based on user preferences. In this case, due to some unusual combination of preferences, Service-A ended making call to Service-B with parameters that Service-B didn’t fully understood. Solution - Write an integration test to cover this scenario. Reasoning - Since this involved 3 different components (User Preferences, Service A and Service B), it should be covered by an integration test. More Reasoning: This is sort of the usual way of thinking about the integration tests - meaning whenever there are multiple services involved, we immediately think about writing an integration test. But is this the right approach specially in microserv...

Microservice architecture paradigm fits well within the Unix’s first principle/philosophy: “Make one program do one thing well”.  We love creating small services which has identity of its own (within a bounded context). These services have their own CI/CD pipelines and provides loose coupling b/w components which significantly improves our ability to deliver code to production. In reality, a workflow can span multiple services. Its very common to find scenarios like below: In the above example, a workflow spans 3 services (Service-1, 2 and 3). Service1 receives the original request. In order to process this request, it GET(s) some data from its dependent services(d1/d2/d3). It further runs some business logic and produces an intermediate response #A. It then makes a state mutable call (POST/PUT/DELETE/PATCH) to Service-B using #A as input and waits for its response. Service-B follows the same paradigm and the workflow ends when all services returns.  The scena...

Microservice architecture comes with a high cost of service dependencies which impacts the overall performance of the system. In a typical setup, when a request hits the service, it first needs to grab data from its dependent apis and after which it will perform more operations and return the final result back to the caller. Dependent apis may further call other apis or talk to a DB in order the return the requested data and hence creating a complex graph of dependencies.  Caching is an effective way to improve the performance of such system. In many cases, you might be able to cache the response of the dependent apis locally and avoid making those extra calls specially if the data is somewhat static in nature. TTL is a great feature which allows you to cache a data for certain duration after which it expires. The pattern looks something like this: First check if the data is cached. If not, get the data from the original source and cache it for future calls. Recently...

Thank you folks for raising some great questions. I am consolidating them here for everyone to benefit! How is your experience using Kubernetes?  >> Kubernetes seems very powerful and some experts would say its the modern day kernel for container orchestration. Internally, we use Rancher which I believe has Kuberneter support but I don’t think we are using it yet. We have an entire SRE team that brings fun delivery tools into the pipeline and they are actively looking into it. What are the preferred languages for writing microservices? Any recommended frameworks? >> It really depends on the type of work. If you are writing heavy ETL pipelines then python is pretty awesome! If its a CPU intensive use compiled languages - golang/.net core/java. If its orchestration style or simple get/put/delete DB operation - use nodejs. All the services should be deployed as container so it doesn’t matter which language is used. In terms of frameworks, I have used aiohttp, flask...

One of the most important selling point of moving towards microservice architecture is the speed of delivery! Today customers can research products, compare reviews and ultimately switch to better products online with unprecedented ease. To survive in this high customer-focused and fast moving industry, we need our teams to deliver software at an accelerated rate and possibly multiple times in a day. For us over the time, our delivery pipeline has also evolved: During the Enterprise days, the release cycle was 6 weeks long. If you missed anything - wait for another 6 weeks. We used home grown scripts to coordinate all the deployments and overall experience was just awful. During the early days of Cloud, we moved to a daily deployments but these were at the end of the day with significant downtime and required lots of coordination with different teams. Today we have deployments on demand. Teams are deploying multiple time in a day with zero downtime! We love Slack at...

Adoption of microservices is not easy and I have had my own share of painful experiences during the initial phase of the transition. In this blog, I will go over some of factors that can be critical to determine the outcome of such transition. Lack of Engineering Culture - What’s an Engineering Culture anyway? Its a culture that practices lean-principles and further enables osmosis style learning for everyone. There is recognition for great work and people are willing to learn from each other. Its a culture where everyone is empowered and feels responsible to participate/evolve/improve all aspects of the SDLC irrespective of their primary role. You can have specialized teams for the different parts but everyones feedback is welcomed. Processes are not followed like a religion but evolved throughout the lifecycle. Adoption of microservices is a steep learning curve for everyone. If an organization doesn’t have the right Engineering mindset, avoid doing microservices. You have ev...

Dark-launches and Feature toggles are two great practices to deploy code faster to production and get early feedback. Combined with microservices architecture, it becomes a very powerful tool and once adopted, there is no turning back! You will be thrilled by the speed with which you can build and ship software in a safe manner! In this blog, I will try to explain what these terms means. Dark Launch Dark launch is a practice where you can test/validate your new functionality in production without having to worrying about disturbing the original workflow. This technique is very useful when writing a new version of the functionality or even adding more functionality that can be triggered based of any existing workflow. The steps to achieve it are very simple: Deploy the new functionality in prod.  Incase if it is new version of an existing functionality and you want to test correctness:  clone the request going into v1-logic —> then call the v1-logic —> clone the...

In my previous blog ( Microservices and Testing - Lessons Learned ), I talked about variation in the testing pyramid and how new flavors of Component Tests (with same good characteristics as those of Unit Tests) have gained momentum and are the primarily source for testing the entire µ-service stack in isolation. In this blog I will talk about how we use the same Component-Test setup to run local Chaos tests!  Microservice architecture is very susceptible to intermittent infrastructure/networking failures which makes Chaos-Engineering a very important aspect for a mature stack. Chaos-Engineering sounds bit chaotic but it is surprisingly easy to do in microservices architecture, specially the local chaos scenarios. To run a full blown Chaos Test, you will need to set aside some dedicated time and bring members from different teams together and run the experiments (Game days!). On the other hand, local chaos scenarios are the ones that you can run locally for your own service sta...