The internet's routing system harbors a fundamental truth that surprises many engineers encountering it for the first time: packets traveling across autonomous systems rarely follow the shortest or fastest paths. This isn't a bug waiting to be fixed. It's the deliberate consequence of architectural decisions made decades ago that prioritized something far more valuable than optimal routing.

The Border Gateway Protocol—BGP—serves as the internet's interdomain routing system, coordinating how approximately 75,000 autonomous systems exchange reachability information. Yet BGP makes no attempt to calculate globally optimal routes. It cannot even express concepts like latency, bandwidth, or path length in its native decision process. Instead, it implements a sophisticated policy engine that allows each network operator to encode business relationships, peering agreements, and local preferences into routing decisions.

Understanding why BGP works this way reveals something profound about large-scale distributed systems. The internet grew from a research project into the backbone of global commerce not despite BGP's limitations, but because of them. The protocol's designers recognized that a network spanning competing commercial entities, sovereign nations, and diverse technical infrastructures could never agree on what optimal even means. They built something more robust: a system that lets each participant optimize locally while maintaining global connectivity.

BGP's path selection algorithm evaluates routes through a strict hierarchy of criteria, and technical metrics occupy surprisingly low positions in that hierarchy. The protocol considers local preference first—a completely arbitrary value that operators configure to express business relationships. A route through an expensive transit provider might carry a lower preference than one through a settlement-free peer, regardless of which path offers better latency or fewer hops.

This design reflects a fundamental insight about commercial networking: business constraints trump technical optimality. When AS 65000 purchases transit from AS 65001, it expects to use that paid capacity before falling back to other paths. When two networks establish a peering relationship, they typically agree to exchange traffic only between their respective customers, not to provide transit to the broader internet. BGP's policy mechanisms make these agreements enforceable at the routing layer.

Consider what global optimization would require. Some central authority would need to collect topology information from every autonomous system, understand the capacity and latency of every link, and compute paths that minimize some objective function across the entire internet. Beyond the obvious scalability problems, this approach founders on a more basic obstacle: network operators will not share detailed internal topology information with competitors or upstream providers.

The internet's commercial structure demands policy flexibility. A content delivery network might prefer routes through specific interconnection points to control costs. An enterprise might route certain traffic through security inspection infrastructure regardless of path efficiency. A national network might implement routing policies that keep domestic traffic within borders. BGP accommodates all these requirements because it treats routing as a policy problem, not an optimization problem.

Early interdomain routing experiments attempted more optimization-oriented approaches. The Exterior Gateway Protocol and early path-vector proposals explored metrics-based routing between autonomous systems. These efforts consistently failed when deployed across networks with conflicting commercial interests. BGP's designers learned from these failures that stability and policy expression matter more than path optimality in a network of competing interests.

Takeaway

When independent entities must cooperate without trusting each other, optimizing for the whole becomes impossible—the best achievable outcome is letting each party optimize locally while maintaining system-wide stability.

BGP operates as a path-vector protocol, meaning it propagates information about the sequence of autonomous systems a route traverses without revealing anything about internal network structure. When AS 65000 advertises a route to its neighbors, that advertisement contains only the AS path and a few attached attributes. The receiving autonomous systems learn nothing about the routers, links, or internal routing decisions within AS 65000.

This opacity is not a limitation but a deliberate design choice with profound implications for optimization. Global route optimization would require visibility into network topology across autonomous system boundaries. A central optimizer—or even a distributed optimization protocol—would need to know link capacities, internal routing costs, and real-time congestion information from every participating network. Network operators guard this information intensely; it represents competitive advantage and potential security exposure.

The path-vector approach enables what networking researchers call information hiding: each autonomous system presents an abstracted view of itself to the outside world. Internal topology changes, capacity upgrades, and routing adjustments remain invisible to external observers as long as interdomain connectivity persists. This abstraction layer allows networks to evolve independently without coordinating changes with thousands of peers.

Information hiding also provides crucial security properties. If BGP propagated detailed topology information, attackers could map network infrastructure with unprecedented precision. The current opacity forces reconnaissance through active probing methods that networks can detect and defend against. Proposals to enhance BGP with optimization-relevant metrics consistently encounter resistance because they would erode this protective abstraction.

Some research efforts have explored cryptographic approaches that might enable optimization without full disclosure—secure multi-party computation for distributed route optimization, for instance. These remain firmly in the research domain. The computational overhead, coordination complexity, and trust requirements make deployment impractical at internet scale. More fundamentally, they solve a problem most network operators don't want solved: operators benefit from information asymmetry that lets them control how traffic enters and exits their networks.

Takeaway

Information hiding in distributed systems isn't just about security—it's what allows independent evolution and prevents coordination costs from overwhelming the system as it scales.

BGP achieves remarkable stability across a network of 75,000 autonomous systems, but this stability comes at significant cost to routing efficiency. The protocol's convergence mechanisms prioritize reaching a consistent global state over finding optimal paths. When network topology changes, BGP's path exploration behavior can cause routing oscillations that persist for minutes—an eternity in networking terms—before settling into a stable configuration.

The minimum route advertisement interval, typically set to 30 seconds for external BGP sessions, exemplifies this trade-off. This timer prevents rapid-fire updates from overwhelming routers and propagating instabilities across the internet. But it also means that better paths discovered during convergence cannot be immediately adopted. The protocol deliberately slows down to maintain stability, accepting suboptimal routing during and after topology changes.

Link-state protocols like OSPF and IS-IS achieve faster convergence and closer-to-optimal routing within single administrative domains. They do so by flooding complete topology information to all participants, enabling each router to compute shortest paths independently. Researchers have periodically proposed link-state approaches for interdomain routing, but these proposals fail the scalability test. Flooding topology updates across 75,000 autonomous systems would generate catastrophic control-plane traffic volumes.

BGP's path-vector approach scales because update volume grows roughly linearly with the number of prefixes and AS path lengths, not quadratically with network size as flooding-based protocols would. This scalability comes from discarding exactly the information that would enable optimization. Each autonomous system makes routing decisions based on limited local knowledge, accepting that globally suboptimal outcomes are the price of keeping the control plane manageable.

The internet routing system thus occupies an interesting position in distributed systems design: it achieves global coordination without global visibility, and reaches stable states without optimizing toward them. Proposals for route optimization must grapple with this fundamental tension. Any mechanism that improves routing quality requires additional state, additional communication, or both—resources that the current system carefully conserves to maintain stability at scale.

Takeaway

Stability at scale often requires sacrificing optimality—systems that converge reliably across thousands of independent actors cannot simultaneously optimize across them.

BGP's design represents a profound engineering compromise: global connectivity without global optimization, stability without central coordination, and policy flexibility without trust requirements. The protocol succeeds precisely because it doesn't attempt to solve the optimization problem. It solves the coordination problem instead.

The internet will never route packets along globally optimal paths because no consensus exists on what optimal means across competing commercial entities. BGP encodes this reality into its architecture, providing mechanisms for local policy expression rather than global metrics optimization. This design choice enabled the internet's growth from research network to commercial infrastructure.

Future internet evolution will likely preserve these characteristics. Even as software-defined networking, segment routing, and programmable data planes transform individual networks, the interdomain boundary will remain a policy boundary. The questions BGP answers—who peers with whom, whose traffic traverses whose network, which paths are permissible under commercial agreements—have no technical solutions. They require the policy machinery that BGP uniquely provides.