API Gateway

A server that acts as the single entry point for client requests into a system of backend services, handling routing and cross-cutting concerns like auth, rate limiting, and TLS on their behalf.

Request routing to the correct service, authentication/authorization, and rate limiting. It also often handles TLS termination, logging/metrics, and response aggregation.

It gives clients one stable, secure endpoint and centralizes cross-cutting logic so each service doesn't reimplement auth, rate limiting, and logging. It also hides internal topology, letting services change without breaking clients.

Kong, NGINX, AWS API Gateway, Apigee, Amazon/Envoy-based gateways, Zuul (Netflix), and Traefik.

No. A load balancer distributes traffic across identical instances (Layer 4/7). A gateway is application-aware and does API-specific work: routing by endpoint, auth, transformation, aggregation. Gateways usually sit behind a load balancer, and often do some load balancing themselves.

The gateway decrypts incoming HTTPS traffic so backends can receive plaintext (or re-encrypted) requests, centralizing certificate management and offloading crypto work from services.

Mapping an incoming request (by path, host, header, or method) to the appropriate backend service — e.g. /orders/* goes to the orders service, /users/* to the users service.

Instead of one general-purpose gateway, you build a dedicated gateway per client type (e.g. a web BFF and a mobile BFF). Each aggregates and shapes backend data specifically for its client, avoiding a bloated one-size-fits-all API and letting teams evolve client-specific concerns independently. Trade-off: more gateways to build and maintain.

The gateway makes several backend calls for one client request and composes them into a single response. It's useful for reducing client round trips (especially on mobile/high-latency networks) and hiding service decomposition. Care is needed: aggregation adds latency (bounded by the slowest call), can couple the gateway to many services, and needs partial-failure handling.

Authentication (verifying identity, e.g. validating a JWT/API key) is typically done at the gateway to avoid duplicating it everywhere, then the verified identity is passed downstream (e.g. as headers). Fine-grained authorization (what this user can do on this resource) often belongs in the services that own the domain logic. In zero-trust designs, services still re-verify tokens (defense in depth) rather than blindly trusting the gateway.

Run multiple stateless gateway instances behind a load balancer across availability zones, keep any state (rate-limit counters, sessions) in an external store, and use health checks and autoscaling. Managed gateways are internally redundant. Also apply graceful degradation and circuit breakers so a partial failure doesn't take everything down.

An API gateway manages north-south traffic (external clients entering the system): auth, routing, rate limiting, external API concerns. A service mesh (sidecar proxies like Envoy managed by Istio/Linkerd) manages east-west traffic (service-to-service inside the cluster): mutual TLS, retries, circuit breaking, and observability between internal services. They're complementary; many systems use both.

It can accept one protocol externally and speak another internally — e.g. expose REST/JSON or GraphQL to clients while calling internal services over gRPC, or bridge HTTP to WebSockets. This lets you keep efficient internal protocols while offering client-friendly external APIs, and lets backends change protocols without breaking clients.

Concerns that are generic and identical across services: TLS termination, authentication, rate limiting/throttling, request/response logging and metrics, CORS, request size limits, IP allow/deny lists, and simple transformations. Business logic and domain-specific authorization should stay out to avoid turning the gateway into a monolith.

It can route by version indicated in the URL path (/v1/, /v2/), a header, or content negotiation, sending each to the appropriate backend version. This lets you run multiple versions concurrently, migrate clients gradually, and even do canary routing (send a small percentage to a new version) — all without clients knowing which backend serves them.

Matters because logic in the gateway creates a shared bottleneck and coupling: every team must coordinate on gateway changes, deploys get riskier, and you drift toward a "distributed monolith." Keep it out by restricting the gateway to generic, declarative policies (routing rules, auth, rate limits) configured as data, not code; push domain decisions (validation, orchestration, authorization) into services or a dedicated aggregation/BFF layer owned by the relevant team; and enforce this via review and platform guardrails. Treat the gateway as infrastructure, not an application.

Set per-route timeouts shorter than the client's, so a hung backend fails fast. Retry only idempotent requests, with a small bounded count, exponential backoff, and jitter to avoid retry storms; budget retries (retry only a fraction of traffic) to prevent amplification. Add circuit breakers that trip when a backend's error/latency crosses a threshold, short-circuiting calls and returning a fallback while the backend recovers. Use bulkheads/concurrency limits per backend so one slow dependency can't exhaust all gateway connections, and shed load (429) under overload.

Call the services in parallel (not serially) so total latency is the max, not the sum. Give each call a timeout; on timeout, return partial results with the missing section marked/degraded rather than failing the whole response (graceful degradation), if the client can tolerate it. Cache slow, less-critical data. Use circuit breakers so a persistently slow service is skipped quickly with a fallback. Distinguish critical vs optional dependencies: fail the request only if a critical one fails. Emit tracing to pinpoint the slow service.

Local per-instance counters over-permit (N instances × limit). Use a shared store (Redis) with atomic operations/Lua scripts implementing a token or sliding-window algorithm keyed by client/API key, so all instances share one budget. To reduce hot-key load on Redis, use local token buckets that periodically reserve tokens from the central budget (a two-tier / batched approach), accepting slight imprecision. Ensure the store is HA and that a store outage fails open or degrades gracefully rather than blocking all traffic.

In zero-trust, no network location is inherently trusted. The gateway authenticates external clients (validating OAuth/OIDC tokens or API keys) and forwards a short-lived, signed identity assertion (e.g. a JWT with claims) downstream. Internal service-to-service calls use mutual TLS (mesh) and their own tokens, and each service still validates the incoming token rather than trusting the caller blindly (defense in depth). Token exchange lets the gateway swap an external token for an internal-scoped one, limiting blast radius if a token leaks.

Downsides: an extra network hop and latency, a central failure/bottleneck point needing HA, operational and configuration complexity, and the temptation to accumulate logic that couples teams. You might skip a heavy gateway for a small monolith or a system with a single client, using a simple reverse proxy/load balancer instead, or rely on client-side discovery + a service mesh for internal traffic. As the system and number of clients/services grow, the centralization benefits usually outweigh the costs.

Version the API and run old and new concurrently. Route by version so existing clients stay on v1 while v2 is available. Use canary routing at the gateway to send a small, growing percentage of traffic (or specific test clients) to v2, monitoring error rates and latency, with instant rollback by shifting the weight back. Provide a deprecation window and communicate it, track per-version usage to know when v1 can be retired, and consider a compatibility/transformation layer so some clients migrate without code changes. Feature flags let you decouple deploy from release.

Use distributed tracing: the gateway generates/propagates a trace ID (W3C Trace Context) that flows through every downstream call, so you can reconstruct the full request path and see where latency/errors occur (Jaeger/Zipkin/OpenTelemetry). Emit structured logs with the correlation ID at each hop, and RED/USE metrics (rate, errors, duration) per route and backend. The gateway is a great place to enforce trace propagation and capture golden-signal metrics because all north-south traffic passes through it. Sample high-volume traces to control cost while keeping errors fully traced.

Yes. For cacheable (e.g. idempotent GET) endpoints, the gateway can cache responses and serve repeated requests without hitting the backend, reducing latency and load. It respects Cache-Control and needs careful cache-key design (and must never cache user-private data in a shared cache).

Rewriting requests or responses as they pass through — adding/removing headers, reshaping payloads, mapping paths, or converting formats — so external clients see a stable, clean API while backends keep their own internal shapes.

Via service discovery: it queries a registry (Consul, Eureka, etcd) or uses platform-native discovery (Kubernetes Services/DNS, cloud target groups) to get the current healthy instances for each service, then load-balances across them. This lets instances scale up/down and move without reconfiguring the gateway, since it always routes to the currently-registered, healthy set.

Orchestration: a central coordinator (sometimes the gateway/BFF or a dedicated service) explicitly calls services in sequence and composes results. Choreography: services react to events independently with no central conductor (event-driven). A gateway naturally does light orchestration for request/response aggregation, but heavy business-workflow orchestration should live in a dedicated service, not the gateway, to avoid coupling and keep the gateway thin.

The gateway can centrally implement Cross-Origin Resource Sharing: respond to preflight OPTIONS requests and add Access-Control-Allow-Origin/-Methods/-Headers to responses, based on a configured allowlist of origins. Handling CORS at the gateway means backends don't each need to, and policy is consistent — but misconfiguring a wildcard origin with credentials is a common security pitfall.

Treat gateway config as code/data: declarative route/policy definitions in version control, reviewed via PRs, and deployed through CI with validation and staged rollout (canary the config). Delegate ownership so each team manages its own routes within guardrails (namespaced config, policy limits) rather than one central bottleneck — often via a control plane (e.g. Kubernetes CRDs like Gateway API, or a service-mesh/xDS control plane) that aggregates per-team specs. Enforce schema validation, automated testing of routing rules, and instant rollback. Avoid manual dashboard edits, which drift and aren't auditable.

The gateway must proxy and hold the connection open rather than treating each message as a request. Considerations: it needs enough connection capacity and memory (each open connection consumes resources), appropriate idle/timeout settings so long-lived streams aren't killed, sticky routing so the connection stays pinned to a backend that holds session state (or a stateless backend with shared state), and graceful draining on deploy (migrate or reconnect clients). Load balancing happens at connect time, so uneven long-lived connections can skew backend load — mitigate with least-connections balancing and periodic reconnection. Auth is done at handshake, and back-pressure/heartbeats keep dead connections from leaking.