Your Uptime Is Worse Than the Friday Brew Rota

There is a system in your office that has never had a postmortem, never needed a platform team, and has been running in production since roughly the invention of the electric kettle. It has survived three restructures, two office moves and the slow death of the office itself. It is the tea round, and it is better architected than the thing that paged you at 3am on Tuesday.

This is not a metaphor dressed up for a conference slide. Look at the round honestly and you'll find leader election, queueing, backpressure, request acknowledgements, dead-letter handling and a failover plan that occasionally involves a saucepan. Look at your own systems and you'll find a Slack channel called #prod-alerts and a hopeful silence.

Anatomy of a system nobody designed

Nobody sat down and specified the tea round. It emerged. Which is exactly how most of your architecture happened too, so the comparison is fairer than it looks.

• Nodes: humans, with wildly variable availability and no health endpoint.
• Shared state: the milk, and how much of it is left. Consistency here is best described as eventual, and occasionally catastrophic.
• Scheduler: whoever last said "anyone want one?" — a leader election nobody votes in and everybody resents.
• Single point of failure: the kettle, and the fuse behind the cupboard that requires a facilities ticket and a small act of God.
• Message queue: the mugs lined up on the counter, in arrival order, roughly.
• Dead-letter queue: the mug with DAVE on it that has been there since Tuesday.
• Observability: the chipped-mug situation, i.e. you cannot reliably tell which drink belongs to whom and must ask, adding latency to the whole operation.

Six mugs, one kettle, no orchestration layer. And yet the drinks arrive. Ask yourself when your deploy pipeline last managed that trick on a Friday.

Four failure modes you will recognise from your own dashboards

The AWOL node. Someone announces "I'll get this round" and then vanishes into a meeting that could have been an email. The work was assigned, never acknowledged, never completed, and no alert fired. Thirteen minutes later a human notices by eye. In your stack that is a hanging request with no timeout, and it is currently holding a connection pool hostage.

The hero. One person gets every round because they're "good at it". Generous on the surface, catastrophic underneath. They go on holiday and the office discovers it has a bus factor of one, disguised as good manners. Your senior engineer who is the only person who understands the billing cron job is the same person, holding the same kettle.

Double delivery. Two people independently start a round. Eight teas arrive for four people. Nobody is upset, but nothing was idempotent either, and half the drinks go cold on the windowsill. Now imagine that with a payments webhook.

Backpressure. The kettle clicks off and six requests land in eleven seconds. The queue drains over forty minutes and the last tea is, politely, a tragedy. Nobody built a rate limiter for the round because nobody thought they'd need one. You built a queue with no bound and called it "elastic".

What the rota knows that your on-call doesn't

Here is the uncomfortable bit. The tea round gets five things right that most engineering teams still argue about in retro.

Ownership is named out loud. The rota is printed, laminated and stuck to the fridge. It is a publicly readable, boring, universally accessible runbook. Your on-call schedule is in a tool that half the team has never opened and the other half have muted.

Requests are acknowledged. "Yes, two minutes." That is a 200 OK. Silence is a failure state, and everyone in the office knows it, which is why the person who says nothing gets asked again.

Capacity is visible. You can see the kettle. You can see the milk level. You cannot see your own queue depth without a VPN, three dashboards and a colleague who remembers the query.

Failover is immediate and undignified. Kettle dies, someone puts a pan on. No change advisory board, no rollback window, plenty of grumbling. It's ugly. It's also twelve minutes, not twelve hours.

The hero rotates. The round isn't a personality. It's a rota. The moment it becomes a personality, the system has become a person, and people go to Center Parcs.

The beermat spec

If you can fit your coordination model on the back of a coaster, you probably understand it. Try these six lines for the tea round, then try them for your incident process.

1. Name the owner of each round, explicitly, in writing.
2. Acknowledge every request, even if the answer is "not yet".
3. Bound the queue. If you can only carry six, say six, and let the seventh wait.
4. Label the payloads — milk, no sugar, the oat one, not that oat one.
5. Keep the runbook where people look during a crisis, which is the fridge, not a wiki with a login wall.
6. Rotate the hero until there isn't one.

The bit where you go and look at your own house

A £9 kettle, a laminated rota and a shared understanding of what "I'll do it" means will out-coordinate most on-call rotas in British software. That is, on balance, an embarrassing sentence to have to write.

So here's the test. Take your incident response, your deploy process or your team's definition of "done", scribble it on a beermat, and hand it to someone who doesn't work in tech. If they can't follow it, the tea round wins again — and the tea round has never once needed a consultant to explain it.

Someone will still say "I'm not bothered" and then order an oat milk flat white with specific temperature requirements. Requirements gathering is hard everywhere. But at least the round ships.