It is often seen that in a system design interview round, the interviewer often favours the kind of questions, which mirror actual on call crisis, which help them analyse your practical instincts in 45- 60 minutes.​

By doing so, they not only test your ability to diagnose production issues but also test how you handle and drive cross-team optimisations without full rewrites.​

As we have talked through the significance of the question, let’s discuss the actual problem statement and how we should break it down and approach it.​

You’re told users report slow twitter timeline loads (>2 seconds vs <500ms SLA). Diagnose bottlenecks and propose fixes without redesigning the entire system. Could be a caching, database, or ranking issue.​

This here is a diagnosis problem rather than an architecture problem, as we are tasked to identify the root cause and not to design the entire system. As the problem states, the root cause could be low cache hits, DB hotspots or ranking issues.​

So we will focus on isolating the bottlenecks and fixing the issue.

​

Ask the Right Counter Questions First

When we come across a vague complaint like timelines are slow, our first move as a senior engineer is to ask sharp counter questions that will help further expose the root cause of the problem.​

Building on this, the first thing before coming up with a solution is to clarify the scope of the problem.​

We have to think of it as a live outage, and for that, we will need data before proposing any solutions, like whether the slowdown is hitting everyone or just high-profile users with a large number of followers.​

Are cache hit rates dipping below 80%?​

Are database queries spiking past 100ms P99, maybe from unoptimized shards or joins? Is there a geographic pattern?​

Now that we have discussed the scope of the problem, let’s move on to the common bottlenecks that could be the root cause of the problem.

Common Bottlenecks

Twitter timelines are a complex assembly line. The process of creating the timeline contains various steps like pulling tweets from hundreds of followed users, filtering them for privacy and ranking them by relevance and then serving them in milliseconds to millions of users at the global level.​

In this complex system, when the timeline load time crosses 2 seconds, it is rarely a single culprit rather, it is usually a combo of familiar suspects in the timeline pipeline.​

As mentioned in our complaint, the causes for this could be a caching, database or a ranking issue, so we will focus on these three and come up with solutions keeping these factors as the scope of the problem, i.e.​

• Cache Misses.

• Database overload.

• Ranking and merge delays.

Cache Misses: When the precomputed feed vanishes

In ideal scenarios, a timeline request hits a precomputed cache (Redis or Memcached clusters), which holds the user-specific timelines for 5–10 minutes. However, if the cache hit rate is dropping below 80%, that means too many timeline compute requests are being made.

​Since the timeline computation process includes scanning users’ recent tweets, which spawns 50–200+ database queries per request, making the responses slower.

​Causes:

This could happen for various reasons, such as short time to live settings that make the precomputed timeline caches expire in 1–5 minutes.

​Another cause that can happen is Thundering herds strike( cache stampede). What happens here, suppose there is a celebrity account with millions of followers, and a popular cached tweet expires, which leads to millions of people requesting the same tweet at once, resulting in overloading the database.

Step-by-step Diagnosis:

1. The first step is to monitor the global cache hit ratio in a monitoring tool like Prometheus. Is it under 80%? Note the trend over hours.

2. While you are monitoring the cache hit ratio, break it down by user type. Check high-follower users. If they have lower hits then this points to a stampede.

3. You can also run distributed traces with Jaeger or Zipkin. We can take a slow request, follow it and see how many cache layers it misses.

4. You should also check the eviction logs, where high rates mean that there is not enough memory.

Fixes :

1. Tuning the TTLs can help fix that dropping cache hit rate. We can set the TTLs to 10–15minutes for the normal users. For active users, we can apply shorter TTLs.

2. Using Redis distributed locks can ensure data consistency during critical write operations, such as adding a new tweet to followers’ home timelines. A distributed lock, typically implemented using SETNX (SET if Not exists), this will ensure that only one worker process can update a specific user’s timeline at a time, which then prevents race conditions.

3. One of the simplest solutions is to scale up and add more Redis shards.

4. Set max memory to allkeys-lru policy. This evicts the least recently used smartly. Test hit rates post-change.

Database Overload: When queries can’t keep up

A database overload is a condition in which a database server receives more requests, queries, or traffic than it can process. This can lead to severe performance degradation, high latency, errors (e.g., “too many connections”), or complete system failure.

Cause:

In the given case of a slow Twitter timeline, one of the main causes can be the overloading of the database. This usually happens in the case of a cache miss, where the user’s timeline is created live. As we know, at peak hours twitter receives around 100k timelines/sec requests, so even a 5% cache miss rate will result in 5000 timelines being made live.

​And to clarify how 5000 request results in a database overload, you need to know that each timeline takes about 50 queries (including ranking process queries), so that a final timeline is created.

That would make the total DB load: 5,000 x 50 = 250,000 QPS.

​And in the case of a celebrity tweet, this scenario is made even worse.

Step-by-step diagnosis:

1. For diagnosing the database overload, the first thing you can do is to find P99 latency that is over 100ms in the query dashboards in Grafana.

2. Then profile the slow queries and run EXPLAIN on fanout SELECTs and then look for seq scans or high row counts.

3. Map QPS per shard so that the load is balanced evenly across shards. You can use tools like Vitness (MySQL proxy) or ProxySQL, which show this live across 1000+ shards.

4. Now simulate the load and reproduce the outage safely using JMeter with celebrity profiles.

Fixes:

1. Building Indexes- Right now, the database queries are scanning millions of tweets to find recent tweets by user X. As we know index is like a book’s index page, so building an index will result in an instant lookup instead of reading every page.

2. Batch Smartly — Instead of 20 separate calls to get tweets from user 1, user 2, and so on, you can ask for all 20 users at once.

3. Add Replicas- right now, a single MySQL is doing everything, i.e. tweet creation and timeline reads. You can clone the database 3–5 times. Where Primary handles writes only, and the Replicas will handle reads only.

4. Async Everything- In the case of a tweet from a celebrity having 50 M followers. The system tries to write timeline caches synchronously. which can block the timeline service for 10+ seconds. You can fix it by changing the database writing process from synchronous to Asynchronous using Kafka.

5. Rebalance Shards- Since a single shard can get hammered by receiving millions of queries per second, while other shards are sitting at 120k queries per second. This can be fixed by doubling the number of database sections so the data can be spread across more shards.

Ranking and Merge Delays: Sorting the Feed Takes Too Long

Ranking and merging delay occurs when the system takes too long to compute and merge the final timeline of a user before showing it on their account.

​It is a computational bottleneck that occurs when the app tries to build a personalised “For You” feed rather than just showing tweets in chronological order.

​Cause:

This happens as a user fetches his timeline, the system must sort 40–50 tweets from different people into an order that is based on the user’s personal preferences. The process of this ordering includes merging, scoring, privacy filtering and ML calculations.

​Before the final timeline is created, the system has to merge 20 streams (one per followed person) into a single list. After scoring the tweets on the params such as likes, replies, and whether the user follows back. And tweets’ freshness is taken into account.

​Then, after applying privacy filters like NSFW? Is the user muted? Or whether the tweet is blocked, the heavy Machine learning happens where computers calculate “topic match” (vectors) that eats up a lot of CPU.

​Step-by-step Diagnosis:

1. Open your tracing tool (Jaeger or Datadog) and then look at one slow timeline request. If Ranking > 100ms, then this is your issue.

2. To check the CPU usage, look at CPU graphs or flame charts. If ML math or matrix multiply is using 50% CPU, then machine learning is slow.

3. Check waiting lines. If the ranking service has long queues (Kafka) that means the service is overloaded.

4. Check garbage collection logs for memory pauses.

Fixes:

1. Async Privacy filters: the first step is to apply a super-fast privacy check based on a probabilistic data structure called a Bloom filter. This gives an instant answer and acts as a high-level gatekeeper that tells the system whether it should even bother running the much slower, more expensive privacy checks. Meanwhile, the verification is done in the background asynchronously, and the privacy team still catches the violations.

2. Smart Heap Merge: In the process of merging 20 tweets from 20 different people a user follows, naive sorting will compare every tweet against every other tweet, which results in hundreds of comparisons. The fix here is to use a heap data structure that only saves the current top 30 most interesting tweets.

3. Cache ML scores: You can precompute the heavy Machine learning process of scoring the tweets and store it in fast memory (Redis) instead of scoring the same popular tweets repeatedly across millions of timelines.

4. Auto-Scaling servers: Kubernetes watches CPU usage, and in case any pod hit 70% CPU, it automatically spins up 10 more pods within 60 seconds.

Conclusion

In system design interviews, questions like fixing a slow Twitter timeline test your system design skills because they force you to move beyond high-level boxes and arrows and solve actual engineering trade-offs.

​By asking this question, the interviewer not only tests your knowledge of Twitter but also tests your ability to handle several core system design pillars.

​When asked these types of questions, don’t just give answers but start the process by asking questions first, as discussed in this blog.

​Before providing any solutions, trace all the possible bottlenecks and common culprits. This blog discusses the three major issues that can cause this slowdown, i.e. Cache misses, Database hotspots and ML- heavy Ranking and merging delays.

​In short, building a fast timeline is not about one magic button, but it’s a methodical process of finding where the data is getting stuck and how to fix the issues by using fast tools like Bloom filters and caches to skip unnecessary work.

A rate limiter plays a vital role in a distributed system. It acts as a defensive traffic control layer that regulates the frequency of requests and prevents resource abuse, which helps in high availability.

A rate limiter not only prevents resources against intentional DDoS attacks but also ensures fair usage so that no single user or bot can monopolise shared resources, which allows all users to get equal services.

In this Article I will be discussing about the significance of the placement of a rate limiter, although in a real-world distributed system, a rate limiter can be placed at various stages, such as Client, CDN, API Gateway, application level, service Mesh, sidecar and at service/ shared datastore, but for the sake of simplicity we will break down the regions into 3 major parts:

• At the client side.

• As a Middleware

• At the server side.

Despite the fact that CDN often sits between the client and the API gateway, I will count it in the client-side region as CDN is the outermost part of a distributed system, and it sees the traffic before it enters the core system and operates without authentication or business context.

The place at which you want to implement a rate limiter basically depends on the priority of what you want to protect in the system.

Why “Where” Matters More Than “How”

When talking about rate limiting, the first thing that comes to mind is rate limiting algorithms (Leaky Bucket, Token Bucket, etc.), but today, the point of discussion is about the placement of the rate limiter.

You would ask why the placement of a rate limiter is so important. The answer to this lies in the three major factors affecting a distributed system, these are: Cost of Infrastructure, Blast Radius during abuse and failure modes under traffic spikes.

Cost of infrastructure

The placement of a rate limiter determines how deep a request can go through a distributed system before it gets rejected. The deeper the request travels, the more resources it will consume along the way. That means the lowest cost will be in the case of placing the rate limiter at the Client or Edge/CDN, as the request will be blocked at the network perimeter itself and can not travel forward using anymore of the resources.

Placing the rate limiter at the service Level will result in the highest cost expenditure, as the request will pass through the gateway and internal network to reach the microservices, ending up using resources at various stages.

Blast Radius during abuse

A rate limiter can be thought of as a primary architectural lever that we can use to control the blast radius. In system design, a blast radius can be described as the extent of damage a system will sustain when something goes wrong.

Therefore, placing the rate limiter at a higher level (Client or Edge/CDN) blocks bad traffic at the network perimeter, providing broader protection. Still, it will have a larger blast radius as the entire system will be affected.

Vice versa, placing the rate limiter at a lower service level will protect only that service and isolate the blast radius to that one service.

Failure modes under traffic spikes

In case of traffic spikes, the placement of a rate limiter will determine the Failure Mode, i.e. how the system fails. When the rate limiter gets overwhelmed by the spike and fails, there are two options to choose from: we can either go Fail-open( let every request go through the system) or Fail-closed (block every request).

The Impact of placing the rate limiter at a higher level (Edge or API gateway) will impact the entire system, resulting in a fail-closed scenario. On the contrary, if we place the rate limiter at a lower level (Service/ Sidecar), the request will only affect that service without disrupting the whole system, resulting in a Fail-Open scenario.

We have discussed why the placement of a rate limiter is vital in a distributed system; the next question arises: where should it be placed? Should rate limiting be implemented at the Client, Edge/CDN, Gateway, service or inside the app? Let’s discuss this next.

What Are You Actually Protecting?

In a distributed system, there is no single right place for a rate limiter. The placement significantly depends upon the specific problem that the rate limiter will handle. Shifting the focus from “how to implement it” to “What you are protecting” will clear the confusion, and the ideal location can easily be found.

There are four major factors that a rate limiter can prioritise, namely Infrastructure, API abuse, business resource and ensuring fair usage. The priority to protect these resources will determine the placement of a rate limiter. Based on these factors, let’s discuss where a rate limiter should be placed:

Rate Limiter at the Client Side

For protecting the infrastructure, the goal here will be to mitigate DDoS attacks and massive traffic spikes. Therefore, the ideal placement of the rate limiter is at a higher level, such as at the client or Edge/CDN (e.g. Cloudflare, Akamai, AWS WAF).

Blocking the traffic at the Client will ensure that the malicious requests will never consume your internal bandwidth, CPU and Load Balancer resources.

Placing the rate limiter here is the most cost-effective implementation. It will provide the largest potential protection, but at the same time, it has the largest risk if misconfigured, i.e. a mistake at the Edge will take down the entire business of that region.

Rate Limiter as Middleware

A rate limiter as a middleware can be implemented after the client and before the backend services. As we move deeper into a system, the middleware rate limiter sits between the incoming request and your actual business logic.

Here, the primary goal of placing the rate limiter as a middleware is to protect the API ecosystem. Unlike the rate limiter placed at Client the middleware lives where the system understands who the user is through JWTs, API Keys or session cookies.

As a middleware, a rate limiter can prevent a single bad client from hogging all the available threads in the service container. The rate limiter as middleware can be placed at the API gateway.

It acts as a Bouncer for a group of microservices. Before any request passes through a service, the gateway checks a central cache (Redis) to see if the user has exceeded their quota.

Since the request has already entered the system, you have already paid some cost to let it enter the network. However, you also saved the money by stopping the bad request from entering deeper into the system and triggering the expensive Database query for a third-party API call.

Rate Limiter at the server side

When implementing a rate limiter at the server side, we are at the deepest level of the system, which is the Application/Service layer and is the last line of defense before a request can consume real resources. The rate limiter is placed inside the backend application, and by the time a request reaches this level, the limiter has full context and knows exactly what the user is trying to do.

At this level, the primary goal is to protect the business resources and infrastructure (DB/CPU). A rate limiter will limit requests and protect specific heavy database queries or downstream microservices that are fragile and cannot scale as fast as the rest of the system.

Being placed at the Lowest level in the system, it provides most surgical failure isolation, for e.g. if the rate limiter implemented at your payment service fails or becomes slow, it has zero impact on your login service.

However, as the request reached the deepest possible level before filtering, it is the most expensive place to reject a request, as the request has already passed through Edge and gateway, which consumes bandwidth and compute along the way.

Conclusion

In this article, we have discussed the three major regions where a rate limiter can be placed in a distributed system. Although a rate limiter can be placed at various levels in the system, in reality, a rate limiter is rarely implemented in one place. In a typical setup, a limiter placed at the client can block rapid button clicks, at middleware, it blocks abusive IPs and enforces per-tenant quotas when it is placed at the server side.

So the places you want to implement these rate limiters significantly depend on what you want to protect. You can balance these placements, architects can minimise the blast radius of failures, and either go for a Fail closed or a Fail openscenario, it can also optimise infrastructure spend and ensure the system remains resilient even under extreme traffic spikes.

A mature distributed system will place rate limiters at all three regions, which we have discussed; combinations of each placement are best suited for a different purpose. When rate limiting is treated as a layered contract rather than a single control, then the systems can fail predictably, recover faster, and remain fair even when traffic is unpredictable.

In a modern-day distributed system, the failure of the network is not a question of “if” anymore, it is about “when” and how to handle the complications arising when the system fails.

One of the major problems faced in the case of a network failure is the “Consensus Problem” where a set of independent nodes must agree on a single true value or the state of a system.

To tackle this situation, various consensus algorithms are used to ensure fault tolerance and to maintain a single source of truth. In today’s blog, I will be discussing the working, limitation and implementation of the two major consensus algorithms, which are Paxos and Raft and also discuss which one is better for what different scenarios.

Although nowadays these consensus algorithms are implemented as a primitive operating at the control plane, they are not given as a feature to a user.

So the question arises, what is the significance of learning these consensus algorithms? If a developer does not directly implement these in a distributed system, then why learn them in detail?

To answer it simply, learning them provides a developer with deep system design insights, which help them in designing and developing reliable systems and troubleshooting even when they incorporate abstract services like Kafka and etcd.

And also learning them will immensely help developers in their interview preparation, as these are considered a must for a senior role to test one’s designing skills.

Now coming back to these algorithms and to discuss the difference between them, let’s first understand each of their workings.

What is the Paxos algorithm in simple terms?

The Paxos algorithm is a consensus principle that mainly focuses on agreeing on a single value among non-faulty nodes in case of a network failure. In Paxos, there are three main functional roles which are taken by the nodes, i.e. Proposers, Acceptors and Learners.

Proposers: These are the nodes that propose a value to the system ( e.g, X=5).

Acceptors: These are the nodes that are responsible for voting and accepting the proposals.

Learners: These are the nodes that do not participate in the voting process, but after the final value has been chosen, they are notified of the value.

The algorithm decides on a single value only after it has majority support. The process can be broken down into two phases, i.e. Phase1 which is prepare and promise and Phase 2

Which is accepted.

Phase 1: Prepare and Promise

Prepare: In this part of the 1st phase, a proposer node selects a unique number (n) and sends a prepare(n) message to the majority of the Acceptor nodes.

Promise: Upon receiving the message, prepare(n) the acceptor node must respond with a promise that if n is greater than any number it has previously seen and will ignore all future proposals if the number is less than n. It also reports the highest-numbered proposal it has already accepted, if any (e.g. promise(1, null)).

Phase 2: Accept and Accepted

Accept Request: When a proposer has a majority of promises from the acceptors, it sends an accept request Accept(n, value) where n is the proposal number and the value is the single true value to be decided among the nodes. This value is proposed by either the proposer node or is the highest numbered value reported by the acceptors, which is achieved through majority.

Accepted: After the Accept request is send the acceptors accept the request and the value if they have not seen a higher n since the last promise. The value is then sent through notification to the Proposers and learners.

Let us understand this through an example.

• Suppose there are two Proposers, P1 and P2, five Acceptors A1-A5, here majority quorum is 3. Now P1 has the value “red”, and P2 has the value “blue”.

• P1 chooses n = 7 and send prepare message to 5 acceptors A1-A5. Acceptors reply with a promise (7, null) since no previous higher value was there.

• P1 gets 3 promises and now proceeds to the accept phase and sends accept(7, “red”) to A1-A5.

• A1, A2, and A3 accept the request and the value “red” is chosen.

• Now, P2 starts with v as “blue”, unaware of P1. P2 pick n=8 and send prepare message prepare(8).

• A1-A3 reply with promise(8, (7, “red”)) stating the previously chosen value i.e. “red”, and A4 and A5 reply with promise(8, null).

• P2 sees the highest previous value as “red” and has to now change its value from “blue” to “red” to maintain a single value as stated by the Paxos algorithm.

• P2 send the accept(8, ” red”) request. A1-A5 all accept the same value without any conflict and this value is further notified to Proposers P1 and P2 and Learners.

What is the Raft algorithm in simple terms?

The raft algorithm is a consensus protocol that mainly focuses on leader election, which effectively provides the same fault tolerance as Paxos. In this, the consensus is reached by breaking down the process into three main sub-problems that are: Leader election, Log replication and Safety.

In the Raft algorithm, every node can initially assume one of three roles, namely Follower, Candidate, or Leader. Initially, all nodes start as Followers. Every Follower expects a periodic “heartbeat” from the leader so that every Follower can know its authority and that it is alive.

If a follower doesn’t hear from the Leader within a fixed time interval, it assumes the Leader has failed. Now the follower becomes the candidate by incrementing its term number and sends RequestVote RPCs to all other nodes.

If the candidate receives a majority of votes from other nodes, it becomes the Leader for that term. In case no candidate gets a majority, the term ends, and a new term begins with a higher term.

After the leader has been elected, it is responsible for handling all client requests.

For replication, the Leader receives a command from a client and appends it to its local log and sends AppendEntries RPCs to all Followers.

Once the Leader receives acknowledgement of receiving the entry from a majority of Followers, it considers the entry committed and notifies the followers to commit the entry as well, then the result of the command from the client is returned.

The Raft Algorithm focuses on Leader election while maintaining a single value or state across all nodes. It ensures that a candidate can only win an election if its log is as up to date as the other majority nodes. To read more about the Raft Algorithm, you can read it here.

What is the difference between Paxos and Raft?

While both Paxos and Raft rely on majority quorum, the main difference between Paxos and Raft algorithms is the approach they adopt to handle fault tolerance and achieve a single state across a distributed system and the complexity of their implementation.

Paxos is considered for its theoretical elegance, but it is also criticised for being notoriously difficult to understand and implement. On the other hand, Raft is much more understandable and can be easily implemented and thus can be more suited for new production systems.

Where Paxos focuses on ensuring a single true value or state across the system through a multiple phase negotiation process between various node roles, such as Proposer, Acceptors and Learners, the Raft algorithm focuses on achieving a single value through a more leader-centric approach in which only the elected leader is allowed to change the state of a system.

Industry Adoption: Where are they used?

Both Paxos and Raft are foundational to modern distributed systems, but their core design principles dictate where they are used in the industry.

Paxos, for its highly specialised foundational infrastructure, is implemented at tech giants. Whereas Raft, for its ease of implementation and operational simplicity, is adopted at various general-purpose distributed systems.

Usually, Paxos needs more resources to manage the complexity of its implementation, so various large technology companies adopt Paxos and it can be found in their core infrastructure.

On the other hand, Raft’s design prioritises understandability, which has led to its wide adoption in the cloud native and database ecosystems. Organization who prioritize operational simplicity and ease of implementation often prefers Raft algorithm over Paxos.

Raft can be found in various prominent systems such as Etcd, cockroachDB and HashiCorp Vault. Whereas Paxos is implemented at Google Spanner, Google Chubby and Apache Cassandra. Amazon DynamoDB also uses a variation of Paxos called Multi Paxos to ensure strong consistency across its distributed nodes.

Conclusion

To conclude this article, even though as a developers we don’t have to implement these consensus algorithms from scratch anymore, but learning about their workings, pros and cons, their implementation and where and how they are implemented in industry can not only massively help a developer to upgrade their system design skill but also assist in their interview preparations, as a candidates with strong understanding of these foundational concepts of system design are highly sought after and can help you get that senior developer role in any MAANG organization.

Over time, a monolithic application undergoes numerous changes, such as the increase in its codebase, the user base also expands with time, and with this growth also come increased complexities like scaling bottlenecks and reliability concerns. Eventually, migrating to microservices becomes an essential step in the process of application growth.

However, the journey of migrating from monolith to microservices itself presents significant challenges. But before discussing more on strategies, let’s first explore the key obstacles and challenges that an organisation faces during the migration process.

Challenges in the migration process

Migrating from a monolith to a microservices architecture is not an easy task. The major issues and hurdles that a team faces upon migrating are how to manage the database. As we know, a monolith system often relies on a single, tightly coupled database, and this in turn makes it difficult to create independent schemas and split the database for each service.

Another issue teams face is that in the case of a newly created microservice that is running parallel to the current monolith system, it has to share and update the same database, which leads to synchronisation issues, race conditions, and conflicting updates. So to tackle this issue, strong data migration strategies and phased rollouts are essential to overcome these hurdles.

And when creating a microservice with a single functionality, the challenge of keeping the main monolith system up and running without any downtime is another one such hurdle the team has to face.

I have outlined some of the major challenges during the migration from a monolith to a microservice, below.

Refactoring the existing codebase

The process of moving from a monolith system to microservices involves refactoring the existing code. This refactoring moves code out of a shared monolith environment until we are able to remove tight coupling by removing shared dependencies and splitting out the business logic to independent services. Refactoring the existing codebase will be time-consuming and somewhat risky as the current monolith code is old and has decreased module functionality over the years.

Data Management

As I have already shared above, one of the toughest challenges teams will face is migrating the data without breaking existing functionality. Splitting the database raises decisions around data ownership and simultaneously managing and ensuring data consistency.

Running microservices in parallel with the monolith creates additional challenges as both the microservices and the monolith will likely need to read and write to the same database, causing conflicts and other synchronisation challenges.

Managing Service Changes without Downtime

The user of the application knows nothing about what works behind the scenes. So in the process of migration, the system must be up and running in all cases and even when the new services are introduced, while decommissioning the old modules. Achieving this zero downtime becomes challenging and requires careful planning and phased rollouts.

Service Communication

In a monolith system, the inter module calls happen within the same process. However, when we migrate to microservices then these calls now turn into network calls. This change in the intercommunication of different services will now lead to the introduction of new complexities, such as possible failures, retries, and using the circuit breaker pattern.

With this in mind teams must carefully plan the communication strategies to ensure reliability without overwhelming the system.

Team Restructuring

Another challenge which an organisation faces in the migration process is of restructuring the team. As new services are introduced, the teams also need to restructure based around different business domains. This shift from working on a single shared codebase to owning individual services has its own hurdles, the teams must adopt DevOps practices and foster greater cross functional collaboration.

Testing and Debugging

In a monolith system the process of testing and debugging is fairly straightforward because all components run in one place. But when we talk about microservices, the task of testing and debugging gets harder in case of distributed transactions and covering integration across services. Same goes with debugging, it also becomes more complex as now the failures may span multiple services,logs or event data stores.

Approaches for Migrating from Monolith to Microservices

Strangler Fig Pattern

As the name suggests, this pattern for converting a monolith to microservices is inspired by the way a strangler fig tree grows around its host tree, which gradually replaces the host tree until it dies. Similarly, in this pattern, the parts of the original monolith application are replaced gradually by the newly created microservices one at a time until the whole architecture is replaced by the microservices architecture.

Implementation: The process starts with identifying and choosing a specific feature or module and then extracting its logic from the monolith and implementing it in a standalone microservice.

Once this microservice is tested and deployed, then the proxy is updated so that all requests for that functionality are routed to the microservice instead of the monolith. Meanwhile, the rest of the requests still go to the monolith.

Parallel Run Pattern

In this pattern, the original monolith and the microservice run in parallel with each other, both serving the request for a specific functionality. By doing so, the microservice gets a chance to get validated against the real-world traffic while keeping the monolith system as a backup mechanism and is useful in case of high-risk migration. And this is the very case where the challenge of sharing and updating the same database arises, as both systems use a common database, which can lead to synchronisation issues.

Implementation: The process for implementing parallel run starts with choosing a well-defined module from the monolith and developing a corresponding microservice that performs the functionality. The microservice is then deployed while keeping the original monolith functionality active as well. Then, a copy of production requests is routed to the microservice, and the monolithic system is kept as the primary system. By doing so, the microservice gets validated. When the microservice produces the same results as the monolith, then the actual user traffic is routed towards it.

Branch by Abstraction

This pattern creates an abstraction layer, which is a common interface that both the old and new implementations can use, which in turn allows the team to switch between them easily during the migration process. This allows the team to incrementally replace parts of a monolith system within the same codebase.

Implementation: The process starts with creating the abstraction layer that hides the underlying implementation details from the rest of the system. All the parts of the existing system now use this abstraction instead of calling the old logic. The new microservice for a functionality is created behind this abstraction layer. Now the shifting of traffic is done gradually from the old implementation to the new one. Once the new implementation is tested and deployed, the old code and the abstraction layer are removed, finalising the migration.

Domain Driven Design

The main focus of Domain Driven Design is to structure the application around the core business domains instead of a technical layer basis. This design ensures decoupling of the monolith system and aligns microservices according to the business domains in such a way that each microservice maps to a distinct area of business responsibility.

Implementation: The process starts with an Event storming workshop where the entire team identifies the domain events, for example, in an e-commerce application, these events are OrderPlaced, PaymentProcessed and InventoryUpdated. This helps everyone to see the big picture and identify how different processes connect.

Next, the functionalities discovered during event storming are grouped into Bounded Contexts, where a set of specific rules and data apply. For every bounded context, a separate database is created by splitting the monolithic database schema. Now you can start implementing one Bounded Context as an actual microservice. Then the newly created microservice can be included in the system through the Strangler Fig pattern or Branch by Abstraction.

Conclusion

After spending years in creating and migrating systems from monolith to microservices, the one thing that I can say with certainty is that the process of migration is a daunting task in itself. The path is filled with many challenges and hurdles, and you will also face challenges while decoupling the tightly coupled code, which is a very complex and time-consuming process and Another major challenge is managing database consistency when running parallel systems. During the process, you will discover a number of hidden dependencies that will necessitate a revision of how your system really works and require careful planning of each step of migration.

You will encounter a time or two where the progress will seem slow and issues such as a data split or decoupling a service will feel like it turns into weeks of analysis or refactoring. However, once you get past those challenges, transformation will be transformational – your architecture will be cleaner, your components will be independent, and scaling your system won’t feel like a battle again.

In the end, a system that is modular, resilient and easily scalable is worth taking the risk, time and challenges faced along the way.

In the last decade, we have all witnessed how fast the modern-day applications have evolved, and with the ever-increasing user bases, these apps cater to millions and billions of users every year. With this rapid increase in the user base, the one crucial thing that can not be negotiated with is the performance of these applications.

One good example is Netflix. Have you ever thought about how Netflix manages millions of play requests at the same time, and yet the user never has to wait for the streaming of their favourite movie?

As developers, we are bound to get curious about the seamless working of Netflix and what tools and techniques it uses so that we can also improve our app’s performance.

Coming back to the case of Netflix, the answer to its flawless performance lies mainly in Caching. Netflix makes use of caching wherever possible and has multiple caching layers, from local device caching to CDN or distributed caching to server-side caching. It uses them all to make the response of that play button click lightning fast.

So in this article, I am going to discuss in detail the types of caches and where and how we can implement them to make our applications faster. And along the way, talk about some common queries that we get while working with caching. For example, when should you use a cache vs just optimising the database?

To answer this simply, you should always focus on optimising the database first, fixing slow queries, and adding proper indexes. And after that, if you are getting huge numbers of Database hits, then it will be the right choice to implement caching. Now we move on to our next topic.

What is Cache?

To define Cache in a simple way, we can say a cache is a high-speed storage space that stores frequently accessed data for its quick retrieval. As the data retrieval process is slow when the data is retrieved from its original slow source, such as disk, Database or remote server, cache provides a faster alternative for getting the data. I will discuss in detail the types of caches and in how many ways they can be bifurcated depending on their place of implementation.

But before that, I want to shed light on the why part of it. So why should it matter to you as a developer?

In totality, Caching will help you to provide better and faster solutions for your project. Caching will reduce the latency as it will serve the responses in microseconds, as compared to milliseconds if the data is fetched from disk or a remote server. It will help in avoiding repeated calls to the database, resulting in lower infrastructure load. With the use of caching, your application can easily cater to millions of users, achieving scalability easily.

With this, we can answer another common query regarding caching and that is: Should I cache everything, or only expensive queries?

And the answer to that is no, you don’t have to cache everything, as it will lead to a waste of memory. A good rule of thumb is that you should cache frequently accessed data along with the queries that are expensive to execute. And please avoid caching highly volatile data.

Types of Caches

We can divide caches into mainly two ways: first, the hardware-level cache and second, caches based on the location where they are implemented.

Hardware-Level cache

Cache at the Hardware level is a small, fast memory storage inside the computer. It stores a copy of the data from the main memory, which is very commonly accessed. Cache memory is placed closer to the CPU to quickly fetch the data. The closer the cache is to the CPU, the faster the data retrieval. There are three types of cache memory:

1. L1 Cache: It is the first level of the cache memory and is closest to the CPU, which is situated inside the processor itself, near the cores. It is very small in size and in the range between 16 and 128KB. Due to its closeness to the CPU, it is extremely fast and low latency.

2. L2 Cache: This is the second-level cache memory, which is placed inside the CPU core. It is slower than the L1 cache but is larger in size. Its memory size ranges from 256KB to several MB.

3. L3 Cache: This is the third level of the cache memory and is situated outside the CPU and is shared by all of the CPU’s cores. It is the last level of Cache before the CPU accesses the RAM. It is slower than L2 Cache but is larger in size, with a range between 4 and 64 MB.

Caches Based on Location

In a distributed system, caches can be used at various layers to enhance the performance of the application. Let’s discuss where and how they are used.

Client-side Cache

1. Browser cache: As the name suggests, the browser cache is the storage space created on the device by your browser that is used for storing resources like images, stylesheets and scripts. So that in case you revisit the site, the browser can load these from the cache rather than downloading them again.

2. Session Storage: It is used to store session-based data within a web browser. The data is stored as a Key-Value pair and persists across the same tab or window, and is cleared when the tab or window is closed.

Service Worker cache

Service worker cache is a local storage that can be programmed for storing web application resources like HTML, CSS and images. The storage mechanism can be fully customised by JavaScript, which helps the developers to implement custom caching strategies. Service worker cache intercepts requests and stores responses that support offline access.

Distributed Cache

Distributed cache is a concept in which the frequently accessed data is stored across multiple servers or nodes. By storing the data in various nodes, a system can be highly reliable, fault-tolerant and have low latency. Some of the common tools used for distributed caching are Redis, Memcached and Hazelcast. These are used to store data such as user session data, API results and Database queries.

CDN Cache

CDN is a mechanism of serving the content to the user that makes use of a network of globally connected servers known as a Content Delivery Network. CDN stores a website’s static data, such as images, CSS, JS, in its closest server to the user. The process goes like this- when a user requests a web page for the first time, the request directly goes to the original server of the website. When the resources are fetched, then the CDN’s closest server(Edge server) to the user also stores these resources so that if in future the user again requests the web page, the request can be served from the CDN’s edge server instead of the Origin server.

Common Queries About Caching

As we have discussed and covered all the basics of caching and its major types, we can now move on to the common confusions faced by developers surrounding caching.

What happens if cached data becomes stale?

This one is the most common cache invalidation problem. To solve it, one can use one of the many cache eviction policies, such as TT(time to live). Cache invalidation is another technique to handle stale data, in this, the cache data is removed as soon as the source of the data changes. Also, while writing the data to the cache, strategies like write-through and write-behind caching must be used to ensure that the Database and cache contain the same data.

How to choose the right cache eviction policy?

Cache eviction is as crucial as storing the data in the cache; choosing the right cache eviction policy mainly depends on your writing strategy used to store the data. The two most commonly used cache eviction policies that you can implement for cache eviction are LFU and LRU.

LFU (Least Frequently Used)

The least frequently used eviction policy removes the data from the cache whose frequency of access is the lowest. This can be beneficial for scenarios where, in the system, the frequency of data matters more than any other aspect.

LRU (Least Recently Used)

The least recently used cache eviction policy uses the criterion of recency rather than any other aspect of the data accessed and removes the least recently used data from the cache. This can be optimum in scenarios where the recent usage is the strongest aspect of usage in the system, as compared with any other characteristic of the accessed data.

What if the cache goes down?

In any distributed system, failure is inevitable. At some point, some services or nodes can crash. So, what can be done in the scenario where a system’s cache mechanism goes down? In this case, the focus should be on the system’s functioning, as it must not completely halt but degrade gracefully. We can do so by implementing a circuit breaker pattern, which ensures further requests are not sent to the cache.

Client, server, or database, where should I cache?

Cache can be implemented at multiple layers, that is, at the client side, server side and at the database. The ideal place to implement it really boils down to the specific needs of the app and the nature of the data. Where client-side caching is ideal for static content that can tolerate some data staleness. The server-side caching is optimum for frequently accessed data that is dynamic in nature. Database caching is effective for reducing repetitive queries to the database, which can result in lightning-fast data retrieval.

Conclusion

In my many years of building scalable systems, the one thing that stands out is how caching proves to be a lifesaver in several instances for me. I have witnessed instant cut times from seconds to milliseconds after implementing caching at the right places. But the other side of the story is that I have also spent nights debugging mysterious bugs that are caused by stale or inconsistent caches. So the takeaway point here is it’s not just about implementing caching, but to know what to cache, at which layer it is ideal to implement it and how to invalidate it safely. So I would suggest you start simple and measure your hit ratios and always design fallbacks. If done right, caching is not just an optimisation but a very effective tool to make world-class, scalable systems with high performance.

Do you know some of the most successful apps like Amazon, Netflix or even Uber, started as a simple monolithic app and migrated to microservices later on when they experienced issues regarding scalability, resiliency and maintaining the app. The famous incident that led Netflix to migrate to microservices happened in 2008 when a major database corruption led to three days of downtime and disrupted their DVD-by-mail service.

However, using a monolithic approach is the fastest and simplest way to launch an application where all the functionalities of the application are written under a single large codebase and is most commonly used by startups to quickly launch their product.

However, as the codebase and functionalities keep growing with time and an increased user base, the complexities of managing the app and teams also increase and eventually frequent system failures occur along with other problems such as scaling the app, slow deployments and resilience challenges.

Signs You Should Consider Microservices

With increasing complexities a monolithic system faces, there come various signs that suggest the right time to convert your monolithic app to microservices. Let’s discuss some of the indications for the immediate migration.

Scalability Issues

There may arise scenarios where only one of the app functionalities, like product search, gets heavy traffic. In this case, the resources for the whole app must be increased just to cater to the increased traffic of a single functionality, wasting resources on the functionalities that do not need them.

You can consider a scenario of a rapidly growing E-commerce site having a monolithic structure, facing a massive increase in some of its functionalities, like product search and recommendation, while other modules, such as user profile and account setting, see minimal traffic. In this case, the whole application has to be scaled up to handle the increased load in some of the modules. However, if the system were operating on microservices, then only specific services should be scaled.

Slow Deployment Cycles

In a monolithic setup, where there is a single large codebase, a small change in one of the features will need the whole application to be tested and redeployed. And eventually, the deployment cycles will become slower and riskier.

Imagine a scenario where a small bug fix in the checkout process requires testing and redeploying of the entire application, making the process of deployment slow. Implementing Microservices would solve this, as only a single service needs to be redeployed instead of the entire application.

Frequent Failures

In a monolithic application where all the modules are tightly coupled, a single bug in a module, say the payment system, can bring down the whole application, which will lead to reduced reliability and broken user trust.

Complex Codebase

With an increase in the codebase of a monolithic app, the modules become so interdependent that it becomes very hard and complex to develop, debug and test further newly added functionalities. This scenario is also known as the “big ball of mud problem”, where nobody knows anymore how the system is working.

In a tightly coupled monolithic system, you can take a scenario of the payment module failing, which can lead to the failure of unrelated features like product browsing or order tracking also. This hurts reliability and user trust. In a microservices setup, failures are isolated, ensuring the rest of the system remains functional even if one service goes down.

Degradation in team productivity

Too much intertwined code creates a bottleneck as several developers are working on the same large codebase, often stepping on each other’s toes, leading to a slow development and debugging process. With a single large codebase and the above-discussed big ball of mud problem, most of the time is wasted on untangling and separating the previously written modules, degrading the team’s performance drastically, and it is an urgent sign that the app needs to be migrated to microservices.

Limitations in Techstack

In a monolithic system, development quickly starts with using the same programming language, framework and database, which initially benefits the app to quickly launch and grow. However, with time, different functionalities of the app require different technologies to scale efficiently, and then the bottleneck is created with the limited tech stack that can be used and becomes a definitive sign that the app needs to be broken down into different microservices to scale efficiently.

Monolithic vs Microservices

When not to Migrate Yet

Migrating from Monolithic to Microservices solves many problems, but not every time. First, there should be some persistent problems your system is facing for you to fix them. You can not fix what isn’t broken yet. So I will discuss some of the conditions where the migration is not needed and will only result in wasted resources and efforts.

Your App Is Still Small and Simple

If your app is still in the early stages of development, then you don’t need to migrate to microservices because that will only add extra complexity. If you’re still building an MVP or serving a modest user base, a monolith is easier and faster to develop, and can be easily tested, and deployed.

Your Team Is Small

If the team working on the project is small, then migrating to microservices will only slow down the process of development and maintenance, as different microservices require separate teams working on them. A small team often works more efficiently in a single codebase where context is shared.

You Lack Automation & Monitoring

One of the biggest benefits of a monolithic system is that it is far simpler to monitor and maintain if you have limited infrastructure. On the other hand, implementing a microservices system requires a solid DevOps maturity that includes CI/CD pipelines, automated testing, centralised logging, and robust monitoring. Without these foundations, managing multiple services quickly becomes chaotic. So if you lack the infrastructure required for microservices, then you don’t need to migrate to microservices yet.

You Don’t Have Clear Boundaries Yet

In case your application’s business logic is still developing and does not have well-defined domain boundaries yet, then the time to migrate to microservices is still not appropriate. Implementing microservices requires the app’s business logic to have clear and separate domain functionalities, and if it does not have that, then jumping into microservices prematurely may lead to poorly split services and messy interdependencies. It’s better to let your monolith mature until natural boundaries become clear.

Conclusion

In my 14 years of experience working as a senior developer, I have worked on both sides of the spectrum, i.e. on one hand I have helped create some robust monolithic systems that serve their purpose well, and on the other hand I have also designed microservices that enable agility and scalability. I’ve also been part of multiple migrations, helping teams carefully break down their monoliths into microservices when the timing was right.

And the one thing that I’ve learned from my experience is that there is no one-size-fits-all answer to the question of choosing between monolithic and microservices. As i have stated earlier, that monolith isn’t inherently bad. In fact, it’s often the smartest way to start. But as complexity, scale, and team size grow, the cracks in the system begin to show prominently, and that’s when microservices become the natural evolution.

So before deciding, step back and evaluate your app, your team, and your goals. Move to microservices not because it’s trendy, but because it truly solves the problems you’re facing.

Imagine a scenario, your newly launched E-commerce app is live with the mega sale, and suddenly, out of nowhere, some of the products go viral. Within minutes, a surge of traffic comes. Orders are getting processed at five times the expected rate, and dashboards are flashing red. Your system is under pressure and is on the verge of failure. The stakes are high if your system lags or worse fails, then you are definitely losing customers and revenue. In this case, what can you do to avoid the failure and keep the website functioning correctly, keeping customers satisfied at the same time?

In this blog, I am discussing five strategies to avoid total failure of the system in case of a sudden traffic surge. So without wasting any time, let's dive deep into the solution part.

Solutions for Handling Sudden Traffic Surges

Auto-Scaling VMs / Containers

Implementing auto scaling through virtual machines and containers is a good starting point in tackling the problem of a sudden surge in traffic. By implementing autoscaling, the system can automatically increase or remove the number of compute instances and containers based on the real-time status of the traffic.

In the cloud environment setups, cloud services such as AWS or Azure automatically calculate the need for resources based on the usage and accordingly increase or decrease the resources as per the usage.

In Kubernetes, autoscaling is implemented through the Horizontal Pod Autoscaler (HPA), which adds new pods when resource usage spikes.

Resource Allocation Shift (Prioritisation)

This one is the most practical and the quickest solution anyone can implement in case of an unknown and sudden surge of traffic. The concept is simple, in case of increased traffic and limited predefined resources, just dynamically shift the allocation of the resources from the less critical processes and services like batch jobs, analytics, background tasks towards more crucial user-facing services such as checkout, payments and API requests.

For example, in case of a traffic surge during a flash sale, you can pause product recommendation jobs and free up compute for checkout and payment services.

In Kubernetes, you can make use of Resource Quotas and Priority Classes to enforce this.

Serverless Architecture

We have heard and used serverless architecture, and it can be used for offloading parts of your application to Functions-as-a-Service, for example, AWS Lambda or GCP Cloud Functions. The beauty of using serverless architecture is that you scale instantly with the demand, whether your system gets 10 requests or 10 million, you do not need to manage the infrastructure, and for this reason, it can be ideal for scenarios of sudden traffic surge.

It is perfect for event-driven workloads like image uploads, notifications, or order confirmations.

Caching

Storing frequently accessed data and serving it to the customers can massively help in alleviating pressure off the system, resulting in less recomputation and reduced database overload.

The idea is simple: store the data and resources in a cache of the products that are most frequently accessed during the traffic surge. It will not only save the system from failing but also boost the system's performance hugely.

Caching can be implemented at multiple levels, such as CDN cache, which is mainly used to store static resources like images, CSS, and JS. Also, caching can be used at the Application level to store user sessions and API responses. Database Cache is another ideal place to implement a cache to store expensive queries cached in Redis/Memcached.

Queues

Using Queues such as Kafka, RabbitMQ, and SQS between the services can help immensely in the case of a traffic surge, as they will help in absorbing the traffic gradually.

How does it help?

Instead of processing millions of requests immediately, by making use of queues, you can enqueue the requests and let consumers process at a safe pace. This could easily prevent the sudden traffic surge from crashing downstream services.

In the case of our e-commerce website, the checkout API can enqueue orders while the background service can process payments.

Subscribe now

Best Practices & Checklist for Developers

A sudden traffic surge can happen anytime to any system without prior notice, so it is advisable to follow some of the best practices we, as developers, must follow in case it happens. It is not a point of discussion to prepare in advance, as the difference between an outage and smooth sailing lies in preparation. Here are some of the best practices that must be followed.

Load Test Before the Surge

You can’t scale what you haven’t tested beforehand, so load testing should be done to stimulate and monitor system behaviour at peak traffic. Load testing simulates peak traffic and uncovers bottlenecks before anything goes down in production.

To do so, you can make use of tools like JMeter, Locust, or k6 to simulate thousands of concurrent users. The key characteristics to keep an eye on while load testing are to test the database under stress, test the caching effectiveness and test if autoscaling triggers as planned.

Set Safe Auto-Scaling Limits

We have earlier discussed how effective autoscaling is in handling the sudden traffic burst. However, if left unchecked, autoscaling can spiral costs, or it may fail to scale fast enough.

One should define min/max thresholds for pods/VMs and test policies under load.

You should set the minimum and maximum limits in such a manner that the minimum is low enough to avoid unnecessary costs, but at the same time, it should be high enough to handle baseline load, and the maximum capping costs while accommodating maximum demand.

Use Priority-Based Resource Allocation

In the solution part of handling the sudden traffic, we have discussed how shifting the resource allocation dynamically in real time can save the system from crashing.

However, priority-based resource allocation can also be done beforehand as a best practice.

We can start by identifying what is critical and what is non-critical workloads in the system and then assigning priorities to them, as not all services are equally important in a surge.

In Kubernetes, using the PriorityClasses and ResourceQuotas would be a good place to start with. We can also pause/defer non-critical batch jobs during a traffic surge. And lastly, monitor core services such as login, checkout, and payments above all.

Implement Circuit Breakers & Retries

When the system is getting a burst of requests, one overloaded service can cause cascading failures. The most practical solution here is to implement circuit breaker patterns and using libraries like Hystrix and Resilience4j.

Along with using a circuit breaker pattern, applying exponential backoff retries for transient errors and implementing fallback functions will definitely help in handling the requests when the circuit is open, for example, returning cached data or a default response.

Conclusion

In my experience, I have seen and experienced firsthand how a sudden traffic surge can bring a system down, resulting in lost business and unhappy customers. The truth is, as much as these situations are stressful, they are also an opportunity to grow and give us the chance to hone our problem-solving ability.

In a system, surges are inevitable and can happen anytime, but in case they happen, the outage part is definitely optional and can be avoided with the use of the right strategies, such as auto-scaling, serverless components, shifting resources, smart caching, and using queues. You can absorb the shock instead of being crushed by it. The key here is to prepare beforehand. Always load-test before you need it, prioritise what matters, and always have a fallback plan.

What is JWT?

A JSON Web Token (JWT) is an open standard (RFC 7519) that defines a self contained way used to securely transmit information as a JSON object by signing it using a secret or a public/private key pair or using RSA or ECDSA. JWTs are compact, URL-safe, and widely used for authentication and authorization in modern web applications, APIs, and microservices.

JWT is a stateless Authentication format in which the server doesn’t need to store user sessions instead, all the necessary claims are embedded in the token itself, as compared with stateful techniques, where the user session should be stored on the servers and a verification lookup is essential to authenticate the user.

The first confusion can arise as to how the authenticity of the user can be validated without the server lookup. For the answer, first let us take a look at the structure of the JWT then it can be understood easily.

Subscribe now

Structure of JWT

The structure of a JWT (JSON Web Token) is made up of three parts that are separated by dots (.), which are :

header.payload.signature

1. Header

2. Payload

3. Signature

Header

The header typically contains two pieces of information , the first one is the type of token (always "JWT") and the second is the signing algorithm used (e.g., HS256, RS256).

{
"alg": "HS256",
"typ": "JWT"
}

Payload

The payload contains the information of an entity or a user and is known as claims, these claims can also include some additional data, such as the timestamp of the token issued and the expiry time of the token.

• Registered claims: These are predefined claims that are optional but recommended, as they provide a set of useful, interoperable claims. These are sub (subject), exp (expiration), iat (issued at) and iss(issuer).
  
  

• Custom claims: These claims are created to share information between parties that beforehand agreed on using them and which are neither registered nor public claims but are app-specific info like role: "admin".

Point to be noted here is that both the header JSON and the payload is Base64Url encoded to form the first and second part of the Web Token.

Signature

The third and the last part of the token is the signature. The signature is created by taking the header and payload, and signing them with an algorithm such as HMAC SHA256 or RSA, and a secret/private key.

This signature ensures and holds the integrity of the token so that it cannot be tampered with. If someone alters the payload, the signature verification will definitely fail.

Now coming back to our first confusion, as to how the verification is done without a server lookup, to simply answer it: when the server creates the token, it takes the header and payload and then uses a signing algorithm (e.g., HS256, RS256, ES256) along with a key (secret or private key) to generate a signature. The final JWT looks like this Base64UrlEncode(header).Base64UrlEncode(payload).Base64UrlEncode(signature).

Now comes the most crucial part and it is the verification part, where the server receives the token. Upon receiving the token, the server takes the first and second part of the incoming token, the header and payload, and re-computes the signature using the same algorithm and shared secret it used when it created the token. Now, if the signature computed matches the signature in the token, this proves that the token is valid and not been tampered with. To clarify further, let’s look at this process of an authentication step by step.

How JWT-based authentication works step by step

To make things easy to understand, I will try to break down this process into various steps that take place when a user logs in and interacts with an application that is secured by JWT.

1. Initially, the process begins when the user logs into the application using its credentials (username and password) across a secure conduit (HTTPS).

2. Next, the server authenticates the user against its database, and if the user is legitimate, the server creates a JWT by hashing the header, payload and signature together.

3. Then the server sends this JWT object back to the client, typically in the response body or as a Set-Cookie header.

4. After the client has the token, it will store the JWT locally, typically in localStorage, sessionStorage or as an HTTP-only cookie.

5. Thus, for every new successive request, the client can include this JWT as a payload in the Authorisation header using the Bearer pattern.

6. After any request the server receives containing a JWT value, the server verifies the JWT and if the token is valid and not expired, the server does as requested and returns an appropriate response. If invalid or expired, the server rejects the request

Types of JWT: JWS vs JWE

As we have already discussed the step-by-step process of authentication and how tokens are generated and the verification is done, it is quite clear that the payload is simply encoded, which leads to another question:

If the JWT payload is just Base64 encoded, can’t anyone read and change the data?

And the answer to this is yes, anyone can read the data in the payload, as we know Base64 is just an encoding, and not encryption. But that does not mean that they can modify the payload and go undetected. As we very well know, the signature part of the JWT is generated using the header, payload, and a secret/private key. And if someone thinks of tampering with the payload, the next step of signature verification will fail, and the token will be rejected. This is why JWTs are safe for integrity, but you should never put sensitive data like passwords or credit card numbers inside the payload unless you use encryption (JWE).

Which brings us back to the types of JWT. There are two types :

1. JWS (JSON Web Signature)

This is the most common form of JWT, and the above process that I have explained is based on JWS. The payload is signed using a secret (HS256) or private key (RS256/ES256). And anyone with the secret or public key can verify that the token has not been tampered with. But as you can notice, the payload is not encrypted and it is only Base64URL encoded, so anyone can read the claims.

So it is best suited for scenarios where authentication and authorization are more important than secrecy.

2. JWE (JSON Web Encryption)

Unlike JWS, here the payload is encrypted in addition to being integrity-protected. Only the intended recipient with the correct decryption key can read the contents. This ensures both the confidentiality of the info that claims contain and the integrity that the token cannot be tampered with.

JWS is widely used in scenarios where it is mandatory to hide the info, as the data can be very sensitive like personal information or financial details, that must be protected inside the token.

Conclusion

Wrapping up this blog post, I would say as a developer, the key here is that it’s not enough just to use JWT but to truly understand how it works and know all its aspects like the differences between JWS and JWE, why the payload is encoded and not encrypted, and the process of the signature verification.

In my 14 years of experience as a developer, I have mentored many junior developers and have seen them facing small to big confusions around JWT, and I have always suggested to them that clearing these queries is essential to avoid misuse that could compromise both performance and security.

We as developers have seen firsthand how much of modern software truly runs on APIs. Over the past 14 years of working as a Senior Developer, I have always focused on the bigger picture while building large scale enterprise applications, and APIs consistently emerge as one of the most critical pieces of building any system. You can imagine them as the connective tissue of applications, which enables systems to talk to each other and scale globally, and can easily evolve with business needs. But as APIs empower communication between the systems, they also introduce a unique set of security challenges.

And it does not matter whether you see it from a Technical architect’s perspective or a developer's, we should always consider some important questions like securing the API while making the API scalable to a million users? Or how do we enforce standards, security, and versioning across the organization?

Emphasizing the security aspects of the API is critical, and in my career, I’ve had the opportunity to handle these challenges and achieve success. I’ve worked on designing APIs that had to survive real-world scale, while also securing the endpoints. Those experiences have shaped my impression of treating APIs as products, and not just interfaces and the importance of securing them.

That’s why I’ve put together these 9 API security best practices, which are not just some abstract theory, but are the lessons distilled from years of building, integrating, and scaling APIs. I will try to capture the dual responsibility of carrying and securely delivering APIs that are technically sound, secure, and future-proof, while also making them intuitive, reliable, and developer-friendly.

So without wasting any time, let’s discuss these 9 API security Best Practices.

1. Use HTTPS Everywhere

While building an API using plain HTTP is not at all secure. It is just like shouting your password in a crowded room. As we know APIs exchange sensitive data such as passwords, tokens, and critical information like payment details, the need for encryption is non-negotiable. That is why using HTTPS is of utmost importance. It protects against eavesdropping and man-in-the-middle attacks. So one of the best practices is to force HTTPS, and enable HSTS (HTTP Strict Transport Security).

For example your banking login PIN is encrypted end-to-end so no one can sniff it over public Wi-Fi.

2. Implement Rate Limiting

Without Rate Limiting APIs are open to various threats. Even a single user can bring down your system by overloading it with fake requests. Think it like a restaurant and without imposing any limits, one greedy client can overload the kitchen. That is why implementing Rate limiting is pivotal as it shields APIs from brute-force logins and DDoS floods. The best practice is implementing Set per-user, per-IP, or per-key thresholds. Queue excess requests.

Take for example the case of using banking apps. In this case we can rate-limit login attempts to prevent credential stuffing.

3. Strong Authentication & Authorization

The concept Authentication and authorization again provides the necessary checks of who is accessing, while Authorization checks what the user can do. Get either of this wrong, and your API is wide open. That is why strong Authentication and Authorization should be implemented to prevent account takeovers and privilege escalation. Some of the best practice is to use OAuth 2.0, JWT, and role-based access control (RBAC).

Take an example of a banking system where a user can view transactions, but only bank admins can modify loan records.

4. Secure API Keys

Securing your API keys is another important aspect we should focus on while building any API. Think API keys as the keys to your house, if you lose them, then anyone can walk into the house, such is the case with the API keys if they are compromised any one can access the system giving attackers free access. The best practice is to store them in secret managers, never in code repos and rotate regularly.

Think of a scenario of a leaked payment API key that can let hackers charge unlimited transactions.

5. Validate and Sanitize Input

Another aspect of API security is validating and sanitizing the inputs. APIs take input from outside which gives attackers a good chance to inject malicious data. That is why it is important to block SQL injection, XSS, and command exploits. One of the ways to do so is to enforce strict input validation and sanitize everything.

A straightforward scenario can be preventing an attacker from sending "DROP TABLE users;" through a query field.

6. Logging & Monitoring

Logging and monitoring are as important as any other aspect of securing the API. If you don’t log, you’re flying blind. By implementing the right Logging and Monitoring techniques, it would help in detecting suspicious activity before it’s too late. So the best practice is to capture requests, errors and anomalies. For this, you can use tools like ELK and Datadog.

A good example of this can be detecting a sudden spike in failed ATM login requests, which may signal brute-force attacks.

7. API Versioning

As with every other thing, the Tech evolves too, and so should your APIs. But breaking old clients overnight is a recipe for chaos. So it is of utmost importance to keep existing apps working while you innovate. For this the best practice is to implement versioning in URLs (/api/v1/, /api/v2/) or headers, and deprecate gradually.

Take the example of Payment gateways, which often run multiple versions for smooth migration.

8. Configure CORS Properly

CORS decides which websites can talk to your API. And if you misconfigure it, the chances of attackers hijacking sessions would increase drastically. So, configuring CORS definitely matters because it could help in preventing malicious websites from exploiting already logged-in users. The Best practice here will be allowing only trusted origins and restricting methods.

For example, in a banking system scenario, your banking API should accept requests from bank.com, not any random domains.

9. Handle Errors Wisely

Handling Errors wisely is of utmost importance. You see, verbose error messages surely help the developers, but also the hackers. Imagine revealing stack traces or DB errors openly, which could hand attackers an easy roadmap. Here, the Best practice is to return generic messages to users and log details for devs. So, instead of exposing DB details, show internal server error.

Final Thoughts

APIs power everything today, from ordering food to streaming movies, and yes, even transferring money in your banking app. Every system implements the communication between them by using APIs. And by the end of this article, it is very clear that without proper security, those same APIs can become the weakest link in your system.

And by following these 9 API security best practices, which include implementing HTTPS and input validation to rate limiting and meaningful error handling, you must ensure that your APIs are safe and protect sensitive data like banking credentials, they are reliably handling high traffic without breaking, and lastly they should be resilient in detecting and stopping attacks before they spread.

Think back to that moment when you opened your banking app. Security is the reason in the first place that you trust it with your money. Start small, apply these tips today, and build APIs your users can rely on tomorrow.

Imagine a scenario, you are working in a big e-commerce platform and the biggest sale of the year is going on and various limited stock products are getting a ton of traffic. Now as you know every distributed system is built with multiple servers working together, and it is not a big deal to encounter a machine failure or a network split.

Coming back to our e-commerce biggest sale situation and a server crashes, now for a specific product, server A thinks 5 items are left and server B thinks 7 are left and both accept the orders simultaneously. This will lead to Overselling leaving to angry customers with broken trust. This is the classic split-brain problem, when distributed systems can’t agree on a single truth.

To ensure that these chaotic situations never happen, we use consensus algorithms to maintain data consistency across all the servers in case of any crashes. While there are many consensus algorithms such as Paxos, PBFT, PoW to name a few, the consensus algorithm I will be discussing today is The Raft Algorithm. But before that let us look at what are the core challenges we face in a consensus algorithm.

The Core Challenges of Consensus

While at first it may seem a simple task to implement a consensus algorithm, but as we try to implement them we get to know that the challenges are real. It is like trying to get a group of friends to pick a restaurant over a spotty phone call where some phones die mid-conversation. Some core challenges of consensus are:

1. Leader Confusion – The situation in which two servers think that they’re the leader, they can accept conflicting client requests. Example: two checkout servers both confirm the last item in stock leading to overselling.

2. Node Failures – There are situations where servers crash, restart, or get partitioned from the network. But the system must keep functioning without losing confirmed transactions, which can be challenging.

3. Consistency Guarantee – There can be system failure but the users shouldn’t see different truths. If one server says “order confirmed”, no other server should say “out of stock.”

Without consensus, distributed systems can easily drift apart, and chaos follows. Raft solves these issues with a clear model which is electing the leader, then replicating the existing log , and afterward safety. Let's discuss them in detail.

How Raft Works?

We have discussed the problem scenarios where the consensus algorithm excels. Now let us understand the working of the Raft algorithm. The first part of its working is the Leader election.

1) Leader election:

In the process of electing a leader initially every Raft cluster starts with nodes as followers. If a follower doesn’t hear from a leader within a certain time limit which are also known as heartbeat messages (for ensuring the health and status of other nodes), it becomes a candidate and requests votes from others with certain conditions such as

1. Each node can vote once in a term.
   

2. If a candidate gets a majority vote in the election process it becomes the leader.
   

3. The leader then starts sending heartbeat messages to reassure followers.
   

This will ensure that there is only one leader at a time, preventing split-brain.

2) Log Replication

Once a leader is chosen, now it becomes the single entry point for client requests. And the log replication process now can go seamlessly. Let’s say a customer buys a product. The request flows like this first the Client sends a Place Order request to the leader. Leader then appends this as a log entry. Then the Leader shares the entry with all the followers. Once a majority confirm, the entry is committed. The leader applies the order then responds “Success” to the client.
This majority rule means even if a minority of servers fail, the system still agrees on the same sequence of events.

3) Safety & Consistency

But there may be a scenario in which the leader crashes mid-process? For cases like these Raft ensures that there are no inconsistencies in the data. It does that by making sure that only a node with the longest, most up-to-date log can become the new leader. This prevents older nodes from undoing already-committed entries. And if there are followers that are lagging behind, they are brought back in sync by the new leader. So even during failures, Raft guarantees that there are no double leaders which avoids the split-brain problem, and that there are No lost confirmations such as a committed order that will not vanish in case of a crash.

Trade-offs & Considerations

As no algorithm is perfect and every algorithm has its strong as well as weak points, Raft is no exception. It makes certain trade-offs to stay simple and reliable. For example, raft focuses on data consistency and partition tolerance over availability, what it means is that raft will rather stop taking new requests than take the risk of conflicting data. It is also very simple to implement this algorithm and is very reliable in what it does. However, it makes certain trade-offs to stay simple and reliable. Understanding these will help you know when to use Raft and when it might not fit.

Let’s talk about the limitations of the raft algorithm:

1. Majority Required – Raft needs a quorum so let’s say in a 5-node cluster, at least 3 must be alive to make it work. But if more than half fail, the system will halt.

2. Latency in Elections – If a leader fails, clients may have to wait until a new election finishes. So in mission-critical apps, this pause can hurt UX.

3. Network Costs –With each entry requiring replication and majority confirmation, the process becomes slower than just writing to a single machine.

Conclusion

While wrapping up this article we can say that consensus is one of the hardest problems which we face in a distributed system. And Raft solved it not by being more complex, but by being more approachable. That’s why it became the go-to choice for engineers which work hard to build resilient infrastructure.

So the next time you launch a Kubernetes pod, discover a service with Consul, or secure a secret in Vault, remember that there’s a Raft cluster silently making sure every node agrees on a single truth and maintaining consistency throughout the system.

In distributed systems, failures are guaranteed but achieving consensus through Raft is how we turn chaos into order.

Subscribe now

We all know Kubernetes is awesome as it removed the big hassle of deploying and scaling of the application while also making it easy to manage them.

But let’s admit it, it can also be confusing as hell at some times. Between Deployments, Services, and Ingress, sometimes it feels like you need a whole new certification just to ship a feature. And things get messy when the deadlines are breathing down your neck.

As a developer with 14 years of experience, I know there are times when we do not have the time to sit and decode every Kubernetes concept just to try something new.

That’s why I’m cutting straight to the point. Kubernetes Blue-Green Deployment is one of the simplest ways to roll out new versions in Kubernetes with zero downtime and a super quick rollback if things go sideways. And you don’t need to become a Kubernetes guru to pull it off.

In this guide, I’ll walk you through the exact steps so you can implement Blue-Green Deployment quickly, saving you those precious minutes. And after that, you can read about the Pros and cons, Best practices, and comparison with other deployment strategies.

Quick Step-by-Step Blue Green Deployment Implementation Guide

Great so let’s jump straight into a practical, implementation of Blue-Green Deployment in Kubernetes with code snippets and clear explanations for each step. I’ll use a tiny demo app (hashicorp/http-echo) so you can try everything locally (minikube/kind) before applying it to prod.

Step 1: The first step is to create the Blue deployment environment. This will be the stable version of your app that is currently running.

apiVersion: apps/v1
kind: Deployment
metadata:
 name: demo-blue
 namespace: bgd
 labels: { app: demo-app, track: blue }
spec:
 replicas: 2
 selector:
 matchLabels: { app: demo-app, track: blue }
 template:
 metadata:
 labels: { app: demo-app, track: blue }
 spec:
 containers:
 — name: app
 image: hashicorp/http-echo:0.2.3
 args: [“-text=BLUE v1”]
 ports: [{ containerPort: 5678 }]
 readinessProbe:
 httpGet: { path: “/”, port: 5678 }
 initialDelaySeconds: 2
 livenessProbe:
 httpGet: { path: “/”, port: 5678 }
 initialDelaySeconds: 2

Step 2: We will now expose this deployment through a Kubernetes service. This service will route all the traffic to the blue pods, making it the active production environment.

apiVersion: v1
kind: Service
metadata:
 name: demo
 namespace: bgd
spec:
 type: ClusterIP
 selector:
 app: demo-app
 track: blue # 👈 initially routes to BLUE pods
 ports:
 — name: http
 port: 80
 targetPort: 5678

Step 3: Now we will apply the Blue deployment and its Service, this will ensure that the pods are running and ready, then uses port-forwarding to route local traffic to the Service. Finally, a curl request which confirms that the Blue environment is live by returning “BLUE v1”.

kubectl apply -f blue-deploy.yaml
kubectl apply -f service.yaml
kubectl rollout status deploy/demo-blue -n bgd
# Test via port-forward:
kubectl -n bgd port-forward svc/demo 8080:80
# In another shell:
curl -s http://localhost:8080
# Expect: "BLUE v1"

Step 4: The next step is to setup our Green deployment which will be the new version of your app, deployed alongside Blue but isolated. We will attach a separate internal-only Service to Green, allowing us to run smoke tests and validation before users ever see it.

apiVersion: apps/v1
kind: Deployment
metadata:
 name: demo-green
 namespace: bgd
 labels: { app: demo-app, track: green }
spec:
 replicas: 2
 selector:
 matchLabels: { app: demo-app, track: green }
 template:
 metadata:
 labels: { app: demo-app, track: green }
 spec:
 containers:
 — name: app
 image: hashicorp/http-echo:0.2.3
 args: [“-text=GREEN v2”]
 ports: [{ containerPort: 5678 }]
 readinessProbe:
 httpGet: { path: “/”, port: 5678 }
 initialDelaySeconds: 2
 livenessProbe:
 httpGet: { path: “/”, port: 5678 }
 initialDelaySeconds: 2

Step 5: We will now create a service for running smoke tests and validation before users ever see it.

kubectl apply -f green-deploy.yaml
kubectl apply -f green-svc.yaml
kubectl rollout status deploy/demo-green -n bgd 
# Port-forward ONLY to green:
kubectl -n bgd port-forward svc/demo-green-internal 8081:80
curl -s http://localhost:8081
# Expect: "GREEN v2"

Step 6: We will now switch the Traffic from BLUE to GREEN by updating the public Service selector (fast, atomic).

# Option A: patch
kubectl -n bgd patch svc demo -p ‘{“spec”:{“selector”:{“app”:”demo-app”,”track”:”green”}}}’
# Option B: edit file and re-apply (service.yaml selector track: green)
# kubectl apply -f service.yaml

To verify:

# Re-port-forward if closed
kubectl -n bgd port-forward svc/demo 8080:80
curl -s http://localhost:8080
# Expect: “GREEN v2”

To monitor:

kubectl get pods -n bgd -l app=demo-app -L track
kubectl logs -n bgd deploy/demo-green

Step 7: In case if issues are detected after switching, you can easily revert by pointing the Service back to Blue pods. This will give you a one-command rollback, which will restore the stability in seconds.

kubectl -n bgd patch svc demo -p ‘{“spec”:{“selector”:{“app”:”demo-app”,”track”:”blue”}}}’
# or re-apply service.yaml with track: blue

Confirm:

curl -s http://localhost:8080
# Expect back to “BLUE v1”

What is Blue Green Deployment

As I have already provided the Implementation steps for integrating Blue Green deployment in Kubernetes for the developers who are looking for a quick integration guide, I can now take my time explaining the concepts and the other aspects of Blue Green Deployment.

So what is Blue Green Deployment in Kubernetes? to simply answer this it can be said that it is a release strategy designed to minimize downtime and risk during application updates. And it is done by maintaining two identical environments which are one live (Blue) and one idle but ready (Green).

The Blue environment is our current deployment which is serving the rel user traffic. And the green environment is the new version that we want to deploy. It is running in parallel but not yet receiving any traffic.

When the time is appropriate for the release of the new version, you can simply switch traffic from Blue to Green. This is usually done by a load balancer or a Kubernetes Service which makes the transition nearly instant.

And the good part is that in case something goes wrong, you can quickly rollback by just flipping the traffic back to the Blue Deployment.

Some Pros and Cons of using Blue Green Deployment

As there are pros and cons with every deployment strategy Blue Green deployment strategy also has its share of pros and cons. Let us talk about the Pros first and then we will proceed with its Cons.

Pros:

• Zero Downtime Releases — In this deployment strategy the new version which is intended for the release i.e. Green environment is already running before the switch, users don’t experience interruptions during deployment.

• Instant Rollback — In case there are errors in the new version, you can immediately redirect the traffic back to the Blue deployment environment which can be done in the matter of seconds restoring the stability.

• Production-like Testing — You can test the Green environment in a real-world environment before redirecting the traffic to it, this will let us catch the issues earlier.

• Simple Traffic Management — In Kubernetes, switching between Blue and Green is as easy as updating a Service selector.

Cons:

• Higher Resource Costs — For implementing Blue Green Deployment strategy there must be two Running environments (Blue and Green) which in turn means double the pods, storage, and sometimes databases resulting in the higher costs

• Database Complexity — For stateful apps, the complexity of maintaining data consistency and availability in both Blue and Green will increase.

• Not Always Necessary — For small services or non-critical apps, Blue-Green may be overkill compared to simpler strategies like rolling updates.

Best Practices for Blue-Green Deployment in Kubernetes

Blue-Green Deployment strategy is very powerful and effective, but to reap all of its benefits, you should be implementing it with care. In Kubernetes you can do so by following these best practices:

1.Automate the Cutover — It is best to automate the processes of running the new version, automated testing and switching the traffic through the use CI/CD pipelines (Argo CD, GitHub Actions, Jenkins.) instead of doing them Manually. By doing so you will decrease the chance of human error.

2. Validate with Readiness Probes –It is of utmost importance to ensure that the Green environment is operational before the traffic switched. Kubernetes readiness and liveness probes ensure that the pods are fully functional and prevents users from hitting a broken environment.

3. Run Smoke Tests Before Switching — It is best to always test the Green environment using internal Services. For this, it is important to validate APIs, key endpoints, and system performance before sending live traffic.

4. Monitor After Cutover — Even after switching, the role of monitoring critical metrics such as latency, error rates, CPU/memory usage with tools like Prometheus + Grafana is pivotal. You have to be prepared to rollback instantly if anomalies occur.

5. Handle Databases with Care — Stateless apps are easy, but stateful ones require backward-compatible schema changes, and always use expand and contract strategy to avoid conflicts.

6. Use Infrastructure as Code — Tools like Helm or Kustomize let you parameterize track: blue/green and manage environments consistently. This reduces misconfigurations.

Use Cases & Real-World Examples

Till now we have discussed the implementation of Blue green deployment, and what it actually is and what are some best practices for implementing it, but all this feels like just a concept and theory. To really understand it one should know the real need for applying this deployment strategy and need to know the real world examples and scenarios where downtime or failed releases are unacceptable:

• E-Commerce Platforms — In any E-commerce platform with a large amount of userbase, even a few seconds of downtime can mean huge revenue loss during a sale event or peak hours, So here implementing Blue Green deployment strategy will ensure smooth rollouts without interrupting customers.

• Banking & Finance Apps — In the Banking and Finance sector. There are several Mission-critical transactions which require maximum reliability. In these scenarios Instant rollback capability makes Blue-Green a safe choice for pushing updates.

• Healthcare Systems — In the Health care systems patient data apps need high availability and compliance. In these critical service sectors Blue Green Deployment allows testing new features in real-like production while guaranteeing uptime.

• SaaS Products — The SaaS product companies offers various subscription-based tools in these scenarios the use of Blue Green environments allows to deploy new features without disturbing thousands of global users.

• Large Enterprises — Organizations with strict SLAs (Service-Level Agreements) often rely on Blue-Green for high-stakes production updates.

Conclusion

To conclude this blog it is safe to say that Blue Green Deployment in Kubernetes definitely provides a reliable, fast, and developer-friendly way to release new versions. By maintaining two environments and switching traffic between them, teams can easily achieve zero downtime, instant rollback, and safer testing in production-like conditions.

As with other strategies this deployment strategy also has its trade-offs, such as higher infrastructure costs and complexity for stateful apps are real, but on the other hand the benefits of it far outweigh them for mission-critical systems. With proper best practices like CI/CD automation, health probes, monitoring, and careful database handling, Blue-Green easily becomes a go-to strategy for production-grade Kubernetes deployments.

For engineers and developers under tight deadlines, this approach can prove to be a lifesaver. It simplifies risk management and gives peace of mind that you can recover instantly if something goes wrong. In short, Blue-Green Deployment lets you ship faster, safer, and with more confidence in Kubernetes.

As we all know that the AI world is evolving at a lightning speed and it is no longer just a buzzword that we discuss at the office cafeteria and for developers, keeping up with AI tech is no longer optional but critical. And falling behind now will lead to outdated skills that will lead to limited opportunities, and be overshadowed by peers who have already embraced AI.

With automation advancing and the introduction of Autonomous AI Agents, the fear of AI replacing jobs is real, but the truth is that those developers who adapt to the change will lead this transformation, not be replaced by it. And missing out will also mean losing the chance to build smarter apps, leverage AI APIs, and design scalable systems that will define the future.

Staying ahead requires learning, experimenting, and exploring breakthroughs like the recent introduction of Nano Banana AI, powered by Gemini 2.5 Flash Image to the AI landscape is one of AI’s latest innovations.

What is Nano Banana AI

Nano Banana AI, which is officially known as Gemini 2.5 Flash Image ( I will later disclose the Interesting backstory of the quirky name Nano Banana ), is Google DeepMind’s latest development in Image generation models. As we know, there are already various powerful Image generation models in the market. The question arises, what’s so different about Nano Banana AI, what does it bring to the table that others can not?

The Gemini 2.5 Flash Image does have various unique, useful features under its belt. To start with, Gemini 2.5 Flash Image provides users with features like Identity Consistency, which keeps the characters the same during multiple renders, it can easily generate and edit an Image with the help of a prompt, it allows multi-turn editing, and last but not the least, it can blend multiple images to generate an image.

The Backstory of the name Nano

Now, coming back to the interesting story behind its eccentric name, Nano Banana. So Nano Banana was started as an informal community-driven nickname given to the Gemini 2.5 Flash Image model. It first appeared on LMArena, which is an AI leaderboard site where various models compete for their AI capabilities. Slowly and steadily, Nano banana started to outperform several popular image-generating models, surpassing them in superior editing capabilities. And soon users started speculating about its origin.

And finally, Google’s CEO, Sundar Pichai, tweeted three banana emojis just before confirming that Nano Banana was indeed Google’s latest Image Generation Model, Gemini 2.5 Flash Image.

Core Features of Gemini 2.5 Flash Image

We now know what Nano banana is and its unique features and the story behind its nickname, but what is its relevance for us developers? To answer it simply, its relevance lies in its developer-first design. Gemini 2.5 is API friendly, which integrates very easily with existing workflows. It enables us to build the apps where visual consistency, semantic understanding and editing flexibility are critical.

It can be well implemented in several projects, like in an E-commerce site to generate product images or simply build a creative storytelling tool. Gemini 2.5 flash Image(Nano banana) can open up new opportunities for innovation.

Now moving on to its core features, earlier in this post, I have already briefly discussed some of the features of Nano banana. Now I will be discussing them in detail.

1. Identity Consistency

To start with, it solves one of the biggest challenges in AI image generation today, and that is of keeping a character, or the objects, and styles consistent across multiple outputs and Nano Banana AI effectively resolves this. Whether it’s for gaming, storytelling, or branding, users can rely on Nano Banana to maintain coherence without random variations.

2. Multi-Turn Editing

Another one of its interesting features is Multi-Turn editing, in which instead of regenerating an image from scratch every time, Nano Banana allows users to initially start with a base image and apply iterative changes to that image, like adding glasses, changing backgrounds, or adjusting lighting, or adding any other object to the image and do all this while preserving the overall quality and identity of the original image.

3. Fusion Engine (Multi-Image + Text Input)

Nano Banana introduces a fusion capability that allows us to combine text and multiple images as input. This allows users to create an image that is the combination of the reference photos and the descriptive prompt given as inputs, which results in producing richer and more controlled images. For instance, a product photo can be reimagined as a futuristic gadget seamlessly.

4. Flash Speed (Optimized for Latency)

Nano Banana is Fast, which processes and delivers the result in seconds, and that is true even for more complex prompts. And this makes it ideal for interactive and real-time applications, such as live editing in design tools or instant product mockups for e-commerce sites.

5. Developer-First APIs

Nano Banana is designed for easy integration. Its APIs can be seamlessly implemented through Google AI Studio and Vertex AI. Gemini 2.5 APIs can be easily plugged into apps, workflows, or enterprise pipelines through minimal lines of code, which ensures flexibility for developers across different domains and makes the apps easily scalable.

6. Content Safety & Post-Processing

Nano Banana provides and also strictly implements features like watermarking, safety filters, and automated quality checks as a built-in Responsible AI. This makes users know the fact that the outputs are not only visually consistent but also compliant and safe for public use.

Simple Nano Banana AI Workflow

As of now, Google has not yet released a white paper with its technical specification. However, experts and the AI community have inferred the key architectural components and their workflow from testing and performance.

From the information available till now and the experience I have gathered from using the Gemini 2.5 Flash Image framework, I can safely imply a basic workflow of Nano Banana.

The architecture of Nano Banana AI is a multimodal generative AI that is unlike traditional image models, which can only accept text as an input prompt. Nano Banana supports multiple input types, such as text instructions and multiple images. Its design uses advanced transformer networks. These networks integrate natural language and visual data to create a rich context for the model to work with.

How to Access Gemini 2.5 Flash Image

So far, we have discussed what Nano banana is, its core features, and a basic Gemini 2.5 Flash Image workflow. The next obvious question is how we can access it. Let’s discuss the ways we can access the Nano banana AI.

As of today, the Gemini 2.5 Flash Image can be accessed through the Gemini API and Google AI Studio for developers.

Developers can access the model through the REST APIs and SDKs that will help in making the app development with Nano Banana flexible and easy to integrate.

In addition to this, an enterprise can also use it through Vertex AI, which provides production-grade deployments and enterprise-level infrastructure while allowing scalability and integration with the broader Google Cloud ecosystem.

Hands-on: Using Gemini 2.5 Flash Image API

And now, coming to the interesting part, let’s talk about how to actually use it. For this, I will be including some code snippets as examples. Google has made it very simple for developers to integrate, experiment, prototype, and scale these via APIs that can be seamlessly implemented in projects.

Let’s begin with the first step to integrate Gemini 2.5 Flash Image (Nano Banana AI) into your application. For this, you need to use Google Gen AI Python SDK. As Google Gen AI Python SDK gives a nice interface for developers to easily integrate all of Google’s generative models into your Python applications. It supports both Gemini Developer API and Vertex AI APIs.

For this we have two main options i.e., Google Gen AI Python SDK or Vertex AI for production deployments.

For the Installation, use the command :

pip install google-genai

Then Import the necessary libraries:

from google import genai
from google.genai import types

And now for creating the client for Gemini Developer API :

from google import genai

client = genai.Client(api_key='GEMINI_API_KEY')

configure your environment by securely loading your API key.

And for creating the client for Vertex API:

from google import genai

client = genai.Client(
vertexai=True, project='your-project-id', location='us-central1'
)

Now that we have set up our client we can now generate the responses with the help of text prompts and images. Let’s implement some of the Gemini 2.5 Flash Image Features:

1. Image generation (text-to-image)

The following code demonstrates how to generate an image based on a descriptive prompt.

from google import genai
from google.genai import types
from PIL import Image
from io import BytesIO

client = genai.Client()

prompt = (
    "Create a picture of an indian sweet dish in a fancy restaurant with a royal indian theme."
)

response = client.models.generate_content(
    model="gemini-2.5-flash-image-preview",
    contents=[prompt],
)

for part in response.candidates[0].content.parts:
    if part.text is not None:
        print(part.text)
    elif part.inline_data is not None:
        image = Image.open(BytesIO(part.inline_data.data))
        image.save("generated_image.png")

Result:

2. Image editing (text-and-image-to-image)

The following example demonstrates uploading base64 encoded images. For multiple images, larger payloads, and supported MIME types

from google import genai
from google.genai import types
from PIL import Image
from io import BytesIO

client = genai.Client()

prompt = (
    "Create a picture of my dog eating an indian sweet dish in a "
    "fancy indian restaurant",
)

image = Image.open("/path/to/dog_image.png")

response = client.models.generate_content(
    model="gemini-2.5-flash-image-preview",
    contents=[prompt, image],
)

for part in response.candidates[0].content.parts:
    if part.text is not None:
        print(part.text)
    elif part.inline_data is not None:
        image = Image.open(BytesIO(part.inline_data.data))
        image.save("generated_image.png")

3. Photorealistic Scenes

For realistic images, use photography terms. Mention camera angles, lens types, lighting, and fine details to guide the model toward a photorealistic result.

from google import genai
from google.genai import types
from PIL import Image
from io import BytesIO

client = genai.Client()

# Generate an image from a text prompt
response = client.models.generate_content(
    model="gemini-2.5-flash-image-preview",
    contents="A photorealistic close-up portrait of an elderly Indian ceramicist with deep, sun-etched wrinkles and a warm, knowing smile. He is carefully inspecting a freshly glazed tea bowl. The setting is his rustic, sun-drenched workshop with pottery wheels and shelves of clay pots in the background. The scene is illuminated by soft, golden hour light streaming through a window, highlighting the fine texture of the clay and the fabric of his apron. Captured with an 85mm portrait lens, resulting in a soft, blurred background (bokeh). The overall mood is serene and masterful.",
)

image_parts = [
    part.inline_data.data
    for part in response.candidates[0].content.parts
    if part.inline_data
]

if image_parts:
    image = Image.open(BytesIO(image_parts[0]))
    image.save('photorealistic_example.png')
    image.show()

Result:

4. Editing Images through prompts

Provide an image and describe your change for example adding or removing an object. The model will match the original image’s style, lighting, and perspective.

from google import genai
from google.genai import types
from PIL import Image
from io import BytesIO

client = genai.Client()

# Base image prompt: "A photorealistic picture of a girl sitting on a study chair, writing notes to study. Soft, natural light from a window."
image_input = Image.open('/path/to/your/sample_photo.png')
text_input = """Using the provided image of the girl, please replace the study chair with a sofa chair."""

# Generate an image from a text prompt
response = client.models.generate_content(
    model="gemini-2.5-flash-image-preview",
    contents=[text_input, image_input],
)

image_parts = [
    part.inline_data.data
    for part in response.candidates[0].content.parts
    if part.inline_data
]

if image_parts:
    image = Image.open(BytesIO(image_parts[0]))
    image.save('girl_studying.png')
    image.show()

Result:

Gemini 2.5 Flash Image vs Midjourney vs DALL-E

With the launch of Nano banana in the image generation landscape, it becomes inevitable to compare it with the already established powerful Image Generation models like Midjourney and DALL-E, to name a few.

From my experience with these 3 Image generation models, I can safely say that each of them has its own set of strengths and weaknesses and there is definitely not one clear winner, but each one of them excels in some performance metrics.

If we talk about Nano Banana, it basically stands out primarily as an image editing tool that excels at consistency, realism, and speed. On the other hand, Midjourney is still the king in pure artistic style.

While DALL-E, which is integrated into ChatGPT, easily provides a strong general-purpose Image generator that is faster than Midjourney, but a bit slower in comparison with Nano Banana. I have thoroughly compared all the aspects of an Image Generation Model and how these three perform at these metrics.

Conclusion

My final thought about Gemini 2.5 Flash Image (Nano Banana AI) is that whether you are experimenting in Google AI Studio or deploying at large scale through Vertex AI, Nano Banana stands out as an Image generation Model for its speed, the ability to perform iterative editing, and nailing features like identity consistency.

It may not be the best at generating highly artistic and stylistic images as compared to Midjourney, but despite that, it has brought many sought-after features to the table.

And it is safe to say that Gemini 2.5 Flash Image is not just another image generator, but it is a toolkit for the next generation of AI-powered applications and staying ahead in this rapidly evolving space means embracing tools like Nano Banana that combine cutting-edge research with real-world usability.