Category: Gaming Systems

How our game server evolved from a monolith to micro game servers

When I started my career at an Indian gaming startup called a23.com, I had no idea I was about to get a front-row seat to one of the hardest problems in online gaming: scale.

I joined a small team – just me, my CEO, and a mentor who doubled as my manager – and I was handed the backend. My job was to build the game server – the piece that keeps everyone in a match synchronized, fair, and connected. In the early days, “scale” meant something modest. We were happy to serve 20 to 100 concurrent players without things falling over.

In the beginning: one socket server to do everything

We started simple. The whole backend was a single socket server, and it did almost everything.

Clients didn’t talk to a web API or a queue — they opened a persistent socket connection straight to the game server and stayed connected for the life of the session. That one server carried the entire weight of the game:

Game events — every move, deal, and turn at the table.

Monetary transactions — wallets, buy-ins, and payouts, all real money.

State and broadcasting— keeping every player at a table in sync.

The loop was beautifully straightforward. A client would send an input over its socket. The server would receive it, process the game logic, update the table state, and broadcast the result to every player seated at that table. Take input, compute, fan out — that was the heartbeat of the whole game.

It was a classic two-tier architecture — clients on one side, a single game server (with its database) on the other

For 20 to 100 players, this was perfect. One process, one connection per player, one place to reason about everything. We could hold the entire system in our heads.

That simplicity was exactly what would later break.

The smartphone wave: when traffic stopped being polite

Around 2010, the ground shifted under us.

Smartphones were suddenly everywhere, and cheap, fast mobile internet — 3G, then 4G — put a gaming client in everyone’s pocket. People who’d never owned a PC were now playing on the bus, in queues, on their lunch break. For the business, this was a dream. For our backend, it was a flood.

Traffic didn’t grow politely. It surged. And our single monolithic server, the one we used to hold entirely in our heads, started to strain. It could still work — but “perfectly” was off the table. One process trying to do game events, money, and broadcasting for an ever-growing crowd was a bottleneck waiting to happen.

We weren’t just trying to survive today’s load. We were getting ready for 10x the traffic.

Step 1: Split the monolith by responsibility

The first move was to stop making one server do everything. We broke the monolith into modular servers, separated by domain— each one owning a clear slice of the system:

Lobby server — players browsing, joining, and matchmaking into tables.

Game server — the actual gameplay: events, table state, broadcasting.

Transaction server— wallets, buy-ins, payouts, and everything money.

Crucially, there was no single gateway in front of these. The client connected to each service directly: one socket to the lobby server for browsing and matchmaking, and a separate socket to the game server once it sat down at a table. Two independent connections, each to the service that owned that part of the experience.

The transaction server sat slightly apart. Instead of talking to it over a direct socket, the game server and transaction server communicated through a message broker. When money needed to move — a buy-in, a payout — the game server published a message and let the broker carry it to the transaction server, rather than blocking gameplay on a synchronous call. This kept the money path decoupled and reliable even if transactions queued up under load, the game kept running.

Now a spike in people browsing the lobby couldn’t drag down a live game, and a heavy moment in gameplay didn’t put financial transactions at risk. Each concern could scale — and fail — on its own.

Step 2: Shard the game servers

Splitting by responsibility bought us room, but a single game-server tier still couldn’t hold all the live tables at scale. So we sharded it.

Instead of one game server for everything, we ran many — and assigned each shard by game type and bet type. A specific game at a specific stake had its own dedicated game servers. Players were routed to the right shard for the table they wanted to play.

Sharding gave us two things we badly needed: horizontal scale (just add more game servers for the hottest game/bet combinations) and isolation (a problem on one shard didn’t ripple across the whole platform).

From garage startup to enterprise: the move to microservices

By 2020, the company I’d joined in a garage wasn’t a startup anymore. It had grown into a full enterprise gaming company. We’d added game after game, and the player base had exploded to 80 million registered players, with up to 200,000 (2 lakh) concurrent players at peak.

Our modular, sharded setup had carried us a long way but the rules of the game had changed again, and this time the pressure came from two directions at once:

Management wanted to optimize infrastructure cost. Running fleets of always-on servers sized for peak traffic meant paying for capacity we didn’t use most of the day.

The business wanted on-demand scaling without manual intervention. When a tournament or a festival spike hit at midnight, no one should have to wake up and spin up servers by hand.

Meanwhile, cloud computing had matured. What used to mean racking our own machines could now be rented, automated, and scaled by API. The timing was right to rethink the foundation.

So we made the leap: containerization with Docker, on a major cloud provider, and a full move to microservice architecture.

Each piece of the system lobby, game shards, transactions, and everything that had grown around them — became an independent, containerized service. That shift gave us the three properties an enterprise platform at this scale can’t live without:

Auto-scaling — services scale up when traffic surges and scale back down when it fades, so we pay for what we actually use. We drove this with CPU-based scaling when a service’s CPU utilization crossed a threshold, the platform automatically added more containers, and removed them again once load eased.

Fault tolerance — if one container dies, it’s automatically replaced; a single failure no longer takes down the experience.

High availability — the platform stays up through deploys, spikes, and hardware hiccups, because no single instance is a point of failure.

This was the moment our “game server” stopped being a server at all. It became a living fleet of micro game servers — spinning up and winding down on their own, healing themselves, and absorbing 200,000 concurrent players as a matter of routine rather than a fire drill.

Step 1: Make the game engine stateless

Auto-scaling sounds great until you ask the hard question: if any container can be killed and replaced at any moment, where does the game state live?

In the old world, a game server held the table state in its own memory. That’s fine when a player is pinned to one server for the whole session — but it’s fatal for auto-scaling. If that one instance dies, the table dies with it. And you can’t freely add or remove nodes if each one is the only place its tables exist.

So the first and most important move was to separate game state from the game engine.

We pulled table state out of the engine’s local memory and into a distributed in-memory cache. State now lived in a fast, shared layer that every game node could read and write — not locked inside a single process.

With state externalized, the game engine itself became stateless.Any game node could now serve any player: it would simply pull the current table state from the cache, apply the move, write it back, and broadcast the update through the message broker to everyone at the table.

That one change unlocked everything else. Because no node “owned” a table anymore, we could add nodes, kill nodes, and reschedule containers freely— exactly what auto-scaling and fault tolerance require.

On top of that, we added a few more pieces to complete the picture:

A matchmaking service — a scalable REST service that the client calls first. Based on matchmaking rules (game type, bet type, seats available, and so on), it picks the right table and returns the game table info to the client. It reads the live picture of tables and seats from the cache to make that decision. Only after this does the client know where to play.

WebSockets for clients— we upgraded the client connections from raw sockets to WebSockets, a better fit for a modern, containerized, cloud-fronted platform.

An API gateway in front of the WebSockets — clients now connect through an API gateway rather than straight to a node. That let us lean on gateway features we’d otherwise have to build ourselves: rate limiting, security, authentication, and routing.

Game engines on a container service — the engine ran as containers on the cloud’s container platform, ready to be scaled and replaced on demand.

Put together, the flow became beautifully elastic. A player first calls the matchmaking REST service, which looks at the available tables in the cache and hands back the game table info. Armed with that, the client opens a WebSocket to the game services for that table, and any available game node can serve them — reading and writing state in the cache, persisting to the database, and fanning out updates through the broker. No node is special. No node is irreplaceable.

Step 2: The hybrid stateful redesign

The stateless design was elegant — but at our peak, elegance got expensive.

At full load we were processing roughly 300,000 (3 lakh) concurrent game events. Because the engine was fully stateless, every one of those events meant a round-trip to the cache and a broadcast through the message broker — and since players at the same table could land on different nodes, nodes had to chatter with each other to stay in sync. Two things started to hurt:

The message broker was spiking, straining under the sheer volume of events flowing through it.

The game engine burned resources on inter-node communication, coordinating state that lived everywhere and nowhere.

Pure statelessness, it turned out, had a cost of its own. So we went looking for a middle path – a hybrid design that kept the operational benefits of microservices but stopped paying the per-event tax.

The core idea: pin one game to one node.

One game, one node. Each game runs entirely on a single game node, which keeps that game’s state in local memory.Gameplay reads and writes happen in-process — no cache round-trip, no cross-node coordination, per move.

Sticky sessions via the API gateway. We map each `gameId` to a gateway cookie, creating a sticky session that always routes a given game on a given node. One sticky session = one game on one node.

Matchmaking hands out the cookie. When the matchmaking service places a player at a table, it returns the cookie along with the table info. The client uses that cookie to connect — through the API gateway — to the same node hosting that game table, every time.

Because all the players of a table now share one node, the engine can use local state for live gameplay. That single change let us dramatically cut message-broker traffic and game-server resource usage the inter-node chatter and constant broadcasting largely went away.

But we didn’t abandon the cache. We still write game state to the distributed cache in the background. Local memory makes gameplay fast; the cache is our safety net – it’s what makes the system fault tolerant, because a game’s state can be recovered even if the node holding it disappears.

A custom scaler service

There was one more problem: CPU is the wrong signal for this workload.A node hosting many low-activity tables and a node hosting a few intense ones can show similar CPU, even though their real load — number of games — is very different. So we dropped CPU-based scaling and built our own.

We added a dedicated Scaler service that owns both scaling and fault tolerance, driven by a metric that actually matters for us: the number of games per node.

Scaling. Game nodes scale out and in based on how many games each node is hosting, not CPU. When games-per-node climbs past our threshold, the scaler adds nodes; when it falls, it removes them.

Fault tolerance. When a node crashes, the scaler spins up a replacement and redistributes the failed node’s games across the remaining nodes in round-robin — rehydrating each game’s state from the cache. Players are reconnected to a healthy node, and play continues.

This hybrid model gave us the best of both worlds: the speed of local state for 300,000 concurrent events, and the resilience and elasticity of microservices for 200,000 concurrent players —without melting the message broker in the process.

Wrapping up: what a decade of scaling taught me

When I look back at the journey — from a single socket server in a garage to a self-healing fleet of micro game servers serving 200,000 concurrent players — the most striking thing isn’t any one piece of technology. It’s that every stage was the right answer for its moment, and the wrong answer for the next one.

A few lessons stuck with me:

Start simple. Embarrassingly simple.That one monolithic socket server was the right call at 100 players. If we’d begun with microservices, sharding, and a custom scaler, we’d have drowned in complexity before we ever shipped. Architecture should match the problem you have, not the one you imagine.

Scale arrives because of the business, not the tech. Every rewrite was triggered by something outside engineering — smartphones, cheap data, 80 million users, a finance team watching the cloud bill. The best architecture decisions came from listening to where the business was heading.

Split by responsibility before you split by scale. Breaking the monolith into lobby, game, and transaction services bought us clarity and isolation long before sharding bought us raw throughput.

There is no “correct” answer on state. We went stateless to unlock auto-scaling, then deliberately walked back toward local state when statelessness got too expensive. Stateless vs. stateful isn’t a religion — it’s a trade-off you re-evaluate as load changes.

Measure what actually matters. CPU looked like a scaling signal until it lied to us. Switching to games-per-node a metric tied to real load made scaling and fault tolerance finally behave.

Decoupling is what lets you sleep. Message brokers, caches, sticky sessions, a dedicated scaler almost every hard-won improvement was really about making one part of the system fail without taking the rest down with it.

If you’re early in your own scaling journey, don’t try to leap straight to the final diagram. Build the thing that works today, watch where it hurts, and let the pain point you to the next architecture.That’s not a failure of planning — that’s how systems that handle hundreds of thousands of players actually get built.

We didn’t design our way to 200,000 concurrent players in one shot. We evolved there, one bottleneck at a time.