Once a system is built from more than a couple of services, discussed in Monolith vs. Microservices, clients need a consistent way to reach the right one, and every service ends up needing the same handful of cross-cutting capabilities. An API gateway centralizes that instead of duplicating it everywhere.

What an API gateway actually handles

An API gateway is a single entry point that sits in front of a system’s backend services, and typically handles several concerns centrally rather than requiring every individual service to implement them separately, as described in AWS’s API Gateway documentation:

  • Routing — directing an incoming request to the correct backend service based on the request’s path, method, or other attributes, so clients interact with one consistent address rather than needing to know the internal location of every individual service.
  • Authentication and authorization — verifying a caller’s identity and permissions once, at the gateway, rather than reimplementing that check inside every backend service.
  • Rate limiting — enforcing request limits per client, discussed in more detail in What Is a Rate Limiter?, to protect backend services from being overwhelmed by any single caller.
  • Protocol translation — accepting requests in one form (a public REST API, for example, potentially negotiated over different HTTP versions) and translating them to whatever protocol or format a specific backend service actually expects internally.
  • Request and response transformation — modifying requests or responses in flight, such as aggregating data from multiple backend calls into a single response a client receives.

How this differs from a load balancer

A load balancer, particularly a layer 7 one, also routes traffic based on request content — which raises a fair question about where the line sits. In practice, a load balancer’s job is primarily distributing traffic across a pool of interchangeable backend instances; an API gateway’s job is broader, encompassing authentication, rate limiting, protocol translation, and API-specific concerns that go beyond simple traffic distribution. Many real architectures use both together: a load balancer distributing traffic to gateway instances, and the gateway itself handling the API-level logic.

How this differs from a service mesh

A service mesh handles similar concerns — traffic management, security, observability — but for traffic moving between internal services, rather than traffic entering the system from external clients. An API gateway is generally the front door for external traffic; a service mesh governs what happens after a request is already inside the system, between services that already trust each other more than an external client would be trusted by default.

When the added hop is worth it

Introducing an API gateway adds a network hop and a new piece of infrastructure to operate, which isn’t free — it’s a real trade-off, not a default every architecture needs from day one. It tends to earn that cost once an architecture has grown to the point where duplicating authentication, rate limiting, and routing logic across many individual services has become a genuine, recurring source of inconsistency and bugs, similar in spirit to when a service mesh starts earning its own operational overhead. For a small number of services, implementing these concerns directly, without a dedicated gateway layer, is often simpler overall.

Key takeaway

An API gateway centralizes the cross-cutting concerns — routing, authentication, rate limiting, protocol translation — that would otherwise need to be duplicated inside every individual backend service, acting as a single, consistent front door for external traffic. It’s a genuine architectural investment that pays off once a system has enough services to make that duplication a real problem, not a default every system needs from the start.

This article explains general API gateway concepts; specific features and capabilities vary by provider and implementation. See our disclaimer.