An alerting system that pages constantly doesn’t make a service more reliable — past a certain point, it makes incidents more likely to be missed, not less, because the people who are supposed to respond have learned, reasonably, to stop fully trusting the alerts.

What alert fatigue actually is

Alert fatigue is the desensitization that happens when engineers are exposed to a high enough volume of low-value, non-actionable, or false-positive alerts that they start responding to all alerts with less attention and urgency — including the genuine ones mixed in among the noise. This isn’t a discipline or attitude problem on the part of the on-call engineer; it’s a predictable, well-documented human response to sustained noise, and it happens to careful, conscientious engineers just as readily as anyone else. Atlassian’s overview of alert fatigue in incident management frames it in exactly these terms: the problem isn’t that engineers stop caring, it’s that a signal-detection system overloaded with noise inevitably gets worse at flagging the signal.

Where the noise actually comes from

A handful of recurring patterns generate most low-value alert volume:

  • Alerting on causes instead of symptoms. Paging because CPU usage crossed a threshold, rather than because users are actually experiencing degraded service, generates alerts for plenty of situations that never actually affect anyone — high CPU that the system handles fine isn’t a problem worth waking someone up for.
  • Thresholds set once and never revisited. A threshold that made sense for a system’s traffic and scale a year ago can be badly miscalibrated for its traffic and scale today, generating either constant false positives or, just as dangerously, missing real problems if it drifted the other way.
  • Duplicate alerts for the same underlying issue. A single root cause that trips five different alerts across five different monitored components turns one real incident into what feels like five separate pages.
  • Alerts nobody has ownership of. An alert with no clear owner or documented response tends to get acknowledged and dismissed reflexively, rather than actually investigated, because there’s no clear expectation of what investigating it would even involve.

The fix: alert on symptoms, not causes

Google’s SRE book discusses this directly in its chapter on monitoring distributed systems: a good alert should page a human specifically when something needs human judgment right now, and should be tied as directly as possible to user-visible symptoms — the four golden signals of latency, traffic, errors, and saturation — rather than to every internal metric that could conceivably, eventually, contribute to a problem. This is also the practical link back to error budgets: alerting tied to whether the error budget is actually being consumed at a concerning rate is inherently more meaningful than alerting on an arbitrary internal threshold that may or may not correspond to any real user impact.

Reducing volume without missing real problems

The goal isn’t simply fewer alerts — a system that pages for nothing is just as broken as one that pages for everything. The actual goal is a higher ratio of actionable, real alerts to total alert volume, achieved through a few concrete practices: consolidating related alerts that share a root cause into one notification instead of several, routing non-urgent issues to a ticket or dashboard instead of a page, regularly reviewing which alerts fired over a given period and whether each one required real action, and retiring or recalibrating the ones that consistently didn’t.

Why this connects directly to incident response quality

An on-call rotation that’s been worn down by chronic noise responds more slowly and less carefully to genuine incidents, covered in How On-Call and Incident Response Actually Work — not because engineers stop caring about reliability, but because sustained noise degrades anyone’s ability to distinguish signal from background. Treating alert quality as an ongoing metric worth actively managing, not a one-time configuration task, is what keeps this from happening.

Key takeaway

Alert fatigue is a predictable consequence of high alert volume with a low proportion of genuinely actionable signal, and it makes real incidents more likely to be missed, not less. The fix is alerting on user-visible symptoms tied to concrete signals rather than every internal metric that could theoretically matter, consolidating related alerts, and periodically reviewing and retiring alerts that consistently don’t require action.

This article explains general alerting and on-call practices; specific thresholds and tooling should be tailored to your system. See our disclaimer.