Most teams configure their load balancer once and never revisit it. They pick round-robin, add a basic health check endpoint that returns 200 OK, and move on. The system works fine—until it doesn't. Then a single degraded instance quietly poisons traffic for thousands of users before anyone notices.

The uncomfortable truth is that load balancer misconfiguration is one of the most common root causes of cascading failures in distributed systems. Not because the technology is flawed, but because the defaults assume a simplicity that production environments never deliver. Your services have different startup times, different resource profiles, and different failure modes. A one-size-fits-all configuration ignores all of that.

This article examines three areas where load balancer configurations routinely fail: health checks that lie about service readiness, algorithm choices that ignore workload characteristics, and shutdown procedures that drop requests mid-flight. Each represents a design decision with long-term architectural implications—and each has a better answer than the default.

The most dangerous health check is the one that always passes. A /health endpoint that returns 200 OK if the process is running tells you almost nothing about whether the service can actually handle requests. The application server might be up, but the database connection pool could be exhausted. The JVM might be running, but a downstream dependency could be unreachable. The container is alive, but the service is functionally dead.

Effective health checks are deep—they verify that critical dependencies are reachable, that connection pools have available capacity, and that the service has completed its initialization sequence. This means checking database connectivity, cache availability, and the status of any circuit breakers. A service that cannot reach its primary datastore is not healthy, regardless of what its process status says. The health endpoint should reflect actual readiness to serve traffic, not merely existence.

There's a nuance here that matters architecturally: you need to distinguish between liveness and readiness. Liveness tells the orchestrator whether to restart the process. Readiness tells the load balancer whether to route traffic to it. Conflating these two concerns is a common mistake. A service that's performing a lengthy cache warm-up is alive but not ready. Killing it would be wasteful. Sending it traffic would be harmful. Kubernetes separates these probes for exactly this reason, and your load balancer configuration should honor the same distinction.

One more consideration: health check intervals and thresholds. Checking every 30 seconds with a single-failure threshold means your system could route traffic to a dead instance for nearly a minute. Checking every 2 seconds with a three-failure threshold detects real failures in about 6 seconds while filtering out transient blips. The right configuration depends on your tolerance for serving errors versus the overhead of frequent checks—but the defaults are almost always too slow for services where latency matters.

Takeaway

A health check that only confirms the process is running is barely better than no health check at all. Design checks that answer the question your load balancer is actually asking: can this instance serve real user requests right now?

Round-robin is the default load balancing algorithm for a reason: it's simple, predictable, and requires no state. It also assumes something that's almost never true in production—that every request costs the same amount of work and every backend instance has the same capacity. The moment you have long-running API calls mixed with fast lookups, or heterogeneous instance sizes behind the same balancer, round-robin starts distributing requests evenly while distributing load unevenly.

Least-connections is a better default for most workloads because it approximates actual load. If one instance is processing a slow database query, it accumulates active connections while faster instances drain theirs. The balancer naturally routes new requests toward instances with more available capacity. This works particularly well for workloads with high variance in request duration—API gateways, services that mix reads and writes, or anything that occasionally calls a slow downstream dependency.

Weighted algorithms add another dimension. If you're running a mixed fleet—say, migrating from older instances to newer ones with more CPU—weighted least-connections lets you assign capacity proportional to actual capability. This is also essential for canary deployments. You can route 5% of traffic to a new version without deploying it to 5% of your fleet. The weight parameter turns your load balancer into a traffic shaping tool, not just a traffic distribution tool.

There's a subtlety that catches experienced engineers: sticky sessions interact poorly with most algorithms. The moment you pin users to specific instances, you've undermined the algorithm's ability to balance load. One instance might accumulate power users with long sessions while another gets lightweight anonymous traffic. If your application requires session affinity, treat it as an architectural constraint that needs its own mitigation strategy—like session externalization—rather than a load balancer toggle you flip and forget.

Takeaway

The right algorithm depends on your workload's variance, not your team's familiarity. If your requests vary significantly in cost or your instances vary in capacity, round-robin is distributing requests fairly while distributing load unfairly.

Deployments, scaling events, and instance replacements all share a common requirement: removing a server from rotation without dropping in-flight requests. This is connection draining, and getting it wrong means users see 502 Bad Gateway errors every time you deploy. In high-traffic systems, even a few seconds of dropped connections during a rolling deployment translates to hundreds or thousands of failed requests.

The correct pattern has three phases. First, the instance signals that it's shutting down—either through a health check that starts returning unhealthy, or through explicit deregistration from the load balancer. Second, the load balancer stops sending new requests to that instance but allows existing connections to continue. Third, after a configurable drain timeout, any remaining connections are forcibly closed. The drain timeout should be long enough to cover your longest reasonable request, but short enough that deployments don't stall waiting for a single hung connection.

WebSocket connections and long-polling add complexity. A standard HTTP request might complete in milliseconds, but a WebSocket connection could persist for hours. If your drain timeout is 30 seconds, those connections get severed. The architectural solution is to design long-lived connections with reconnection logic on the client side, and to use a drain timeout that accommodates your transactional requests while accepting that persistent connections will need to reconnect. Clients that hold long-lived connections should always be prepared for disconnection.

One pattern that's often overlooked is pre-stop hooks in container orchestration. In Kubernetes, there's a race condition between the pod receiving a termination signal and the endpoint being removed from the service. Adding a short sleep in the pre-stop hook—typically 5 to 10 seconds—gives the control plane time to propagate the endpoint removal before the application starts its shutdown sequence. Without this, the load balancer can still route traffic to a pod that's already shutting down. It's a small configuration detail with outsized impact on deployment reliability.

Takeaway

Zero-downtime deployment isn't about deploying faster—it's about removing instances gracefully. Every server removal is a coordinated handoff, and the drain period is what makes the difference between seamless and broken.

Load balancer configuration isn't a one-time setup task—it's an ongoing architectural decision that shapes how your system behaves under stress. The defaults are designed for simplicity, not for the complexity of real production workloads.

The principles are consistent across all three areas: reflect reality, not assumptions. Health checks should verify actual readiness. Algorithms should match actual workload characteristics. Drain periods should accommodate actual request lifecycles.

Revisit your load balancer configuration with the same rigor you'd apply to your database schema or your API contracts. It's infrastructure that encodes assumptions about your system—and wrong assumptions fail silently until they fail catastrophically.