Scaling WebSockets in Production: What Breaks After the First Server

I built a real-time chat platform on Node.js and Socket.IO that ran perfectly on a single container. Messages delivered instantly, presence was accurate, typing indicators felt snappy. Then I put a second container behind a load balancer and roughly half the messages vanished.

Nothing was actually broken. The architecture had simply been holding an assumption I never wrote down: that every connected client shares one process's memory. That assumption is worth understanding in detail, because it is the root of nearly every WebSocket scaling problem.

Why the second server breaks everything

A WebSocket is a long-lived TCP connection pinned to exactly one process. When Alice connects, her socket lives on server A and only server A can write bytes to it. When Bob connects and the load balancer sends him to server B, server B has no handle to Alice's socket. So when Bob emits a message to the room they share, server B iterates over the sockets it knows about — which does not include Alice — and delivers to nobody.

This is not a bug you can patch in the message handler. The room membership registry itself is per-process in-memory state, and you have two processes.

The fix: move fan-out to a shared bus

The durable answer is that servers stop being the source of truth for who is in which room. Instead, every server publishes outbound events to a shared message bus and subscribes to events from it. Redis pub/sub is the standard choice because it is already in most stacks and its latency is well under a millisecond on a local network.

import { Server } from "socket.io";
import { createAdapter } from "@socket.io/redis-adapter";
import { createClient } from "redis";

const pubClient = createClient({ url: process.env.REDIS_URL });
const subClient = pubClient.duplicate();

await Promise.all([pubClient.connect(), subClient.connect()]);

const io = new Server(httpServer, {
  // Skip the long-polling upgrade dance when you control the clients.
  transports: ["websocket"],
});

io.adapter(createAdapter(pubClient, subClient));

With the adapter installed, `io.to(room).emit(...)` publishes to Redis, every server receives it, and each one delivers to whichever sockets it happens to own. The application code does not change at all — which is the point. Fan-out is infrastructure, not business logic.

Presence is a harder problem than fan-out

Message delivery is solved by the adapter. Presence — knowing who is currently online — is not, and this is where most implementations quietly stay broken.

The naive approach increments a counter on connect and decrements on disconnect. This drifts permanently, because processes do not always get a disconnect event. A container killed by OOM, a node drained during a deploy, or a network partition all leave phantom users online forever. Within a week your 'online now' count is fiction.

The correct model is that presence is a lease, not a flag. A user is online because they recently asserted they were, and that assertion expires on its own.

const PRESENCE_TTL_SECONDS = 45;

// Refreshed on connect and then on a heartbeat well inside the TTL.
async function touchPresence(userId: string, socketId: string) {
  const key = `presence:${userId}`;
  await redis
    .multi()
    .sAdd(key, socketId)
    .expire(key, PRESENCE_TTL_SECONDS)
    .exec();
}

// A crashed server never runs this — and that is fine, the key expires.
async function dropPresence(userId: string, socketId: string) {
  const key = `presence:${userId}`;
  await redis.sRem(key, socketId);
}

async function isOnline(userId: string) {
  return (await redis.sCard(`presence:${userId}`)) > 0;
}

Note that presence is a set of socket IDs rather than a boolean. A user with a laptop and a phone has two sockets; treating presence as a boolean makes closing one tab mark them offline everywhere. The set also self-heals: if a server dies, its socket IDs linger for at most one TTL window and then the key expires entirely.

Pick the heartbeat interval at roughly one third of the TTL. With a 45-second TTL, clients heartbeat every 15 seconds, which tolerates two consecutive dropped heartbeats before a user is falsely marked offline.

Backpressure, or why one slow client stalls a server

This one is subtle and it took me a production incident to internalise. When you write to a socket whose peer is not reading — a phone that walked into an elevator, a throttled mobile connection — the data does not disappear. It queues in the process's send buffer. Emit a high-frequency stream to a few hundred such clients and you are holding hundreds of megabytes of undelivered payloads in heap.

The symptom is confusing: memory climbs, garbage collection pauses lengthen, event-loop lag spikes, and healthy clients start seeing delays. The cause is entirely on the write side.

const MAX_BUFFERED_BYTES = 1_000_000; // ~1 MB per socket

function safeEmit(socket: Socket, event: string, payload: unknown) {
  // Socket.IO exposes the underlying engine transport's buffer.
  const buffered = socket.conn.transport.writable
    ? socket.conn.transport.socket?.bufferedAmount ?? 0
    : Infinity;

  if (buffered > MAX_BUFFERED_BYTES) {
    // The client cannot keep up. Dropping is better than dying.
    metrics.increment("ws.backpressure.dropped");
    return false;
  }

  socket.emit(event, payload);
  return true;
}

Authenticate during the handshake, not after

A pattern I see constantly is accepting the connection and then waiting for an `auth` event to identify the user. This leaves a window in which an unauthenticated socket is connected, consuming a file descriptor and able to emit events. Under a trivial connection flood, that window is the whole attack.

io.use(async (socket, next) => {
  try {
    const token = socket.handshake.auth?.token;
    if (!token) return next(new Error("unauthorized"));

    const claims = await verifyJwt(token);
    socket.data.userId = claims.sub;
    next();
  } catch {
    next(new Error("unauthorized"));
  }
});

Rejecting in middleware means the connection is torn down before it is ever registered. Combine it with a per-IP connection rate limit at the edge and the flood surface mostly closes.

One caveat that bites people: JWTs expire, but a WebSocket opened at 09:00 is still open at 17:00 holding claims that went stale hours ago. Either re-verify on a timer and disconnect on expiry, or keep socket sessions short-lived and let clients reconnect.

What to put on a dashboard

Request-rate and CPU graphs tell you almost nothing about WebSocket health, because the traffic is idle-heavy and bursty. These four signals are the ones that actually predicted incidents for me:

• Event-loop lag — the earliest warning that a server is saturated, and it moves before CPU does.
• Connections per process — should be roughly even across the fleet; a skew means the load balancer is not rebalancing after a deploy.
• Reconnection rate — a sustained spike means clients are being dropped somewhere, often an idle timeout on a proxy you forgot about.
• Redis pub/sub round-trip latency — when the bus degrades, every room broadcast degrades with it.

That third one deserves emphasis. Most managed load balancers and reverse proxies close idle connections after 60 seconds by default. If your heartbeat interval is longer than that idle timeout, every client silently reconnects on a loop and you will chase the phantom for days. Set the ping interval below the smallest timeout anywhere in your path.

The short version

Horizontal scaling of real-time systems is not really about WebSockets. It is about noticing which state you assumed was global when it was only ever per-process. Room membership, presence, and rate-limit counters are all that kind of state.

1. 1Put fan-out on a shared bus so any server can reach any client.
2. 2Model presence as an expiring lease keyed by socket, never a boolean flag.
3. 3Bound your write buffers and drop stale live data rather than queueing it.
4. 4Authenticate in handshake middleware, before the connection is registered.
5. 5Alert on event-loop lag and reconnection rate, not CPU.

Get those five right and the second server is a non-event — which is exactly how scaling should feel.