We did not pick one cache — we mapped each workload to CAP
On a live gaming platform, someone always asks: “Hazelcast or Redis?”
As if one logo must win the whole cluster.
We did not choose that way. After modularizing game engines — lobby, game, transactions on their own paths — we needed recoverable game state when a node dies, coordination across JVM services, and high-volume leaderboard reads that could tolerate a little lag. Hazelcast and Redis each fit a different CAP shape.
This article is Hazelcast and Redis — not vs. How we use a remote Hazelcast distributed cluster for replicated table state and failover, and Redis for leaderboards. CAP is the lens; live tables, pinned game nodes, and money kept off the tick are the examples.
Architecture, to us, is not picking logos. It is design, prototypes, and naming tradeoffs out loud. Choose C or A because the business can live with the downside — CAP is a lens, not a bible.
CAP in one minute
| Letter | Plain meaning |
|---|---|
| C — Consistency | Every read sees the latest write (or you get an error) |
| A — Availability | Non-failing nodes keep answering |
| P — Partition tolerance | System keeps running when the network splits |
P is not optional at scale. Under partition you pick C (refuse stale / fence / error) or A (keep serving, maybe stale).
| If the business says… | Lean toward | Gaming example |
|---|---|---|
| “Wrong money or double owner is unacceptable” | C (refuse / fence / error) | Table ownership, routing on failover |
| “Empty screen or long outage hurts more than slightly stale data” | A (keep serving) | Leaderboard top 100, presence hints |
| Workload | CAP lean | Tool |
|---|---|---|
| Live table state + failover | C when JVM or grid member fails | Hazelcast IMap + CP |
| Leaderboard top 100 | A — empty board worse than stale rank | Redis ZSET |
| Wallet / payout | Durable C | DB + queue |
Same company, same platform — different maps, different clusters, different backup settings.
Our model: pinned node + IMap replica
We pin one game to one game node. That node is the active writer — pot, seats, turn, timers — single owner, local memory, fast tick.
Every state-changing action (fold, call, raise, timer) also replicates a snapshot to Hazelcast `IMap` on a remote cluster with sync backup. If the pinned node dies, a new node loads that snapshot and resumes — same pot, same turn.
| Layer | Role |
|---|---|
| Pinned game node (local) | Active writer during play — every fold/call/raise mutates local state first (fast tick) |
| Hazelcast IMap (sync backup) | Recovery copy — replicate snapshot on each state change; new node loads table if pinned node dies |
| CP lock + fencing | Only one game node may resume the table after failover |
| Table → node routing (IMap) | Where clients reconnect after failover |
| Queue + DB | Buy-in / payout — durable money truth, not per-action IMap |
Normal action: local mutate → IMap.set(tableId, snapshot) → broadcast
Game node dies: CP lock → new node IMap.get → hydrate → clients reconnect
oney still flows queue + DB — not per-action cache ledger.
Hazelcast: remote distributed cluster (not embedded)
Dedicated six-member grid. Game / lobby / transaction services are clients only.
| Model | What it means | What we did |
|---|---|---|
| Embedded | Each game JVM is also a Hazelcast member | No — different scale lifecycles |
| Client → remote cluster | Game services are HZ clients; grid is dedicated | Yes — our model |
[ Game node A — active writer ] ──client──┐
[ Game node B — failover target ] ──client──┼──► [ Hazelcast cluster ]
[ Lobby service ] ──client──┘ game state IMap (sync backup)
routing IMap, CP locksWhy remote cluster: scale game fleet independently; blast radius; tune sync backup on grid nodes; game JVM churn does not reshape partitions.
Jobs we gave Hazelcast
| Job | Why Hazelcast (remote cluster) |
|---|---|
| Replicated game state (IMap) | Every state-changing action → snapshot to IMap with sync backup; new game node loads table if pinned node dies |
| Table → node routing | Central IMap; all services know where each table lives |
| Ownership on failover | CP subsystem + fencing — only one node may resume the table |
| Soft metadata | Presence, session hints — rebuildable; async OK |
CAP justification: losing a live money table mid-hand, or two nodes serving different pots, is unacceptable → sync backup on game-state `IMap`, CP + fencing on failover.
What we did not do: two writers without fencing wallet ledger in HZ async-only zero-backup on game-state maps.
Redis: leaderboards (AP-leaning)
| Job | Why Redis |
|---|---|
| Global / seasonal leaderboards | ZSET — score-ordered ranks, top-N in one structure |
| High read volume | Simple, fast, operationally familiar |
| Ranking display | Product tolerates seconds of staleness across replicas |
Leaderboards are AP-friendly: show rank #6 when true rank just became #5 beats an empty board. Redis ZSET, high reads, replica lag OK for display.
What we did not do: Redis as sole payout source; linearizable global #1 on every read full match simulation in Redis.
| Workload | CAP preference | Choice | Why |
|---|---|---|---|
| Live table / pot / turn | Strong C per match | Game node + HZ IMap | Local active writer; replicate snapshot to sync-backed IMap for failover |
| Table → node routing | C on failover | Hazelcast | Sync backup on dedicated HZ members; game nodes are clients |
| Ownership after node death | CP (no split-brain) | Hazelcast CP | One successor; fence old owner |
| Buy-in / payout handoff | Durable, ordered | Queue + DB | Not cache-as-ledger; gameplay stays off money path |
| Leaderboard / ranking | A over strict C | Redis | ZSET top-N; brief staleness OK; high read QPS |
| Presence / soft hints | Eventual OK | HZ async or TTL | Rebuildable; not money truth |
When things fail
Game node goes down
1. Pinned node dies mid-match
2. One successor acquires CP lock (fences old node)
3. ‘IMap.get(tableId)` → rebuild local state
4. Routing updated → players reconnect
5. Resume or void by product rules
CAP: C on failover — we paid sync replication during play.
Hazelcast grid member goes down
1. HZ member loss → backup promotes → short rebalance
2. Game clients retry `get`/`set`
3. Replicated game state survives on promoted copy
Avoid: state living only in one JVM with no recoverable `IMap` copy.
Anti-patterns
- One cache for everything — CAP differs by workload
- Redis Redlock for table ownership — we used Hazelcast CP + pinned node
- Hazelcast for every leaderboard — Redis ZSET won for AP serving
- Async / zero-backup on game-state `IMap`
- Strong consistency everywhere — unnecessary for presence and ranks
Wrapping-Up
Hazelcast and Redis — not vs.
Hazelcast — remote cluster: replicated game state in `IMap`(sync backup), routing, CP on failover. Active writer on pinned node; recoverable copy on the grid.
Redis — leaderboards: available top-N, briefly stale OK.
CAP: pick C or A per workload. Architecture: design, prototype, name tradeoffs. Sync backup costs latency — we paid it where failure consistency mattered. Replica lag costs perfect ranks — we paid it where availability mattered.
Draw the workload on CAP first. Then place Hazelcast, Redis, and the game server.
if your platform debates “Hazelcast or Redis” as one winner — what workload would you map first?
Leave a Reply