Modern network architectures rarely rely on a single path between two endpoints. Data center fabrics, service provider backbones, and enterprise WANs all deploy multiple parallel links to increase aggregate bandwidth and provide redundancy. The question is not whether to use multiple paths, but how to distribute traffic across them intelligently.

Equal-Cost Multi-Path routing, or ECMP, is the mechanism that makes this possible at the forwarding plane. When a router has multiple next-hops with identical routing metrics toward a destination, ECMP selects among them on a per-flow basis, aiming to spread load without reordering packets within a conversation.

The mechanics sound straightforward, but the engineering behind ECMP involves subtle trade-offs. Hash algorithms must be deterministic yet diverse. Multi-stage topologies can suffer from polarization. And when paths have unequal capacities, the equal-cost assumption breaks down entirely. Understanding these details is essential for designing networks that actually achieve the load distribution their topology promises.

ECMP path selection is fundamentally a hashing problem. When a packet arrives and the router finds multiple equal-cost next-hops in the FIB, it computes a hash over selected packet header fields and uses the result modulo the number of paths to pick an egress interface. The hash function must be fast enough to run at line rate and diverse enough to spread flows evenly across available paths.

The choice of hash input fields determines the granularity of distribution. A 5-tuple hash covering source IP, destination IP, protocol, source port, and destination port produces per-flow load balancing, keeping all packets of a TCP or UDP connection on the same path. Simpler implementations may hash only on Layer 3 headers, which is less granular but avoids the cost of parsing transport headers on every packet.

Consistency within a flow is not optional. If two packets of the same TCP connection take different paths with different latencies, they may arrive out of order, triggering unnecessary retransmissions and degrading throughput. The deterministic nature of the hash ensures that identical header tuples always map to the same path, provided the set of available next-hops remains stable.

This last caveat matters in practice. When a link fails and ECMP recomputes its next-hop set, naive modulo hashing remaps most flows to different paths. Resilient hashing and consistent hashing techniques minimize this disruption by ensuring that only flows previously assigned to the failed path are redistributed, leaving other flows undisturbed.

Takeaway

Determinism is the quiet virtue of ECMP. The same hash on the same headers must always yield the same path, or the abstraction of a stable flow collapses.

In multi-stage networks such as Clos fabrics, ECMP can fail in a way that pure single-hop analysis misses. If every router in the topology uses the identical hash function with the identical seed, packets that hash to the first path at stage one will also hash to the first path at stage two, and so on. The result is that only a subset of the theoretically available paths carries any traffic.

This phenomenon is called hash polarization. A network designed with sixteen equal-cost paths across two spine layers might end up using only four or eight of them, with the rest sitting idle while the active paths congest. The topology looks correct on paper, but the forwarding behavior concentrates flows onto narrow slices of the fabric.

The standard mitigation is hash seed randomization. Each router independently seeds its hash function, often with a value derived from its own identifier or a locally generated random number. This ensures that even if two routers use the same underlying algorithm, their outputs diverge for identical inputs, breaking the correlation between stages.

Another technique varies the specific header fields or bit positions each router uses. Some vendors expose configuration knobs to select which fields feed the hash, allowing operators to introduce diversity manually. In symmetric fabrics, verifying that polarization has been avoided requires actual traffic measurement, not just configuration review, because the pathology hides until real flows reveal it.

Takeaway

Identical algorithms at every hop produce identical decisions at every hop. Diversity in hashing is not a nice-to-have but a structural requirement for multi-stage load distribution.

Standard ECMP assumes all paths are equivalent, but real networks often have parallel links of different capacities. A site connected by a 100 Gbps link and a 40 Gbps link should not receive equal shares of traffic, yet classical ECMP would distribute flows evenly and saturate the smaller link long before the larger one filled.

Weighted ECMP addresses this by assigning each next-hop a weight proportional to its capacity. Instead of a single entry per path in the forwarding table, the router installs multiple entries for higher-weighted paths. A path with weight three appears three times in the hash bucket table, while a path with weight one appears once, biasing the selection accordingly.

Implementations vary in how they compute and distribute weights. BGP link bandwidth extended community allows routers to advertise the capacity of upstream links, letting downstream routers derive appropriate weights automatically. In interior routing, protocols like IS-IS and OSPF traditionally require equal costs, though extensions and controller-driven approaches now enable unequal-cost distribution.

The granularity of the weight table matters. Coarse tables cannot represent fine capacity ratios accurately, so a 100:40 split might be approximated as 5:2 or 3:1. This introduces small distribution errors that accumulate under load. Modern silicon supports large hash bucket tables specifically to allow precise weight representation without sacrificing hash quality.

Takeaway

Fair distribution across unequal paths is not equal distribution. Matching load to capacity requires the routing plane to know what each path is actually worth.

ECMP is one of those mechanisms that appears simple in specification but demands careful engineering in deployment. The hash function, the fields it consumes, and the way weights are represented all affect whether a network delivers its designed capacity.

The failure modes are subtle. Polarization hides in symmetric configurations. Unequal capacities silently bottleneck on the smallest link. Flow remapping under failure amplifies transient disruption. None of these problems announce themselves in configuration output.

Sound ECMP design treats hash diversity, weight granularity, and resilience under change as first-class requirements. Measure actual distribution, not assumed distribution, and the parallel paths in your topology will finally behave like the parallel paths in your diagrams.