In 1999, two engineers faced an unusual problem. Oil pipelines stretched across remote terrain, dotted with sensors that needed to report pressure and flow. The only link back to headquarters was a satellite connection that was slow, unreliable, and billed by the byte. Every wasted character cost real money.

Andy Stanford-Clark of IBM and Arlen Nipper designed a messaging system lean enough to squeeze through that narrow, expensive pipe. They called it MQTT. Today, descendants of that idea quietly carry messages between your thermostat, your doorbell, factory robots, and billions of other devices. You never see this network, but it shapes almost every 'smart' thing you own. Here's how it works, and why its design choices matter.

Consider what the web was built for. When your browser loads a page, it wraps each request in hundreds of bytes of extra information: language preferences, cookies, browser details. For a laptop plugged into the wall, that overhead is invisible. For a soil moisture sensor running on two AA batteries, it's a slow death.

Every byte a device transmits costs energy, and radios are among the hungriest components in any gadget. MQTT's answer was radical minimalism. Its smallest message header is just two bytes. CoAP, a protocol standardised by the Internet Engineering Task Force in 2014, takes a similar approach, using a compact four-byte header and skipping the lengthy handshakes that heavier protocols demand. A sensor can wake up, say what it needs to say, and drift back to sleep in a fraction of a second.

Clayton Christensen would recognise this pattern. By conventional measures, these protocols weren't better than the web's tools. They did less. But doing less was exactly what a new market needed. Devices that once required wall power or frequent battery changes could now run for years, opening doors to sensors in places nobody could easily reach.

Takeaway

Sometimes the breakthrough isn't adding capability but stripping it away. A technology that does less can often go places the more powerful version never could.

Imagine a temperature sensor in a greenhouse. In a traditional setup, every system that cares about that reading, the heater, the vent controller, the farmer's phone app, a data logger, would need its own direct connection to the sensor. Add a new consumer and you'd have to reconfigure the device. Multiply that across thousands of devices and the result is a tangle nobody can manage.

MQTT solves this with a model called publish-subscribe. The sensor simply announces its reading to a central hub, called a broker, under a label like 'greenhouse/zone3/temperature.' Anyone interested subscribes to that label. The sensor has no idea who's listening, and it doesn't need to. The broker handles delivery to everyone who asked.

This separation is why connected systems can grow so easily. Facebook adopted MQTT for its Messenger app in 2011, partly because it kept phones connected without draining batteries. Factories can bolt on new analytics tools without touching a single machine on the floor. The model works more like a radio station than a phone call. The publisher speaks into the room, and the room decides who hears. That difference lets networks expand without redesigning what's already there.

Takeaway

Systems scale best when their parts don't need to know about each other. Removing the need to know who's listening is what lets a network grow without breaking.

Back to those remote pipelines. Satellite links dropped constantly, and a missing alarm about pipeline pressure could be catastrophic. Yet resending every message several times would blow the data budget. The designers needed a way to match reliability to importance.

MQTT offers three levels of quality of service. Level zero is fire and forget: send once and hope it lands, which is fine for a reading that updates every few seconds anyway. Level one guarantees delivery at least once, though duplicates may slip through. Level two guarantees delivery exactly once, using a four-step exchange. That matters when a duplicate could cause harm, like a command to open a valve. CoAP achieves something similar by marking messages as 'confirmable,' requiring an acknowledgement before the sender moves on.

There's another clever feature called the last will and testament. When a device connects, it can leave a message with the broker, to be published only if the device disappears unexpectedly. Silence itself becomes information. A sensor that suddenly loses power still manages to say, in effect, 'something has gone wrong with me.' Engineers no longer have to wonder whether a quiet device is calm or dead.

Takeaway

Not every message deserves the same effort. Good systems match the cost of reliability to the cost of failure, rather than treating everything as equally urgent.

MQTT began as a fix for an expensive satellite bill. Over two decades later, it's an open standard running in cars, hospitals, power grids, and homes. Its story echoes a familiar pattern: technologies built for overlooked niches often end up becoming infrastructure.

As billions more devices come online, the quiet conversations between them will matter more than the gadgets themselves. The most transformative technologies are often the ones nobody notices, until they stop working.