1. Introduction
MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for constrained devices and low-bandwidth, high-latency networks. Due to its minimal overhead, reliable message delivery, and efficient distribution, it has become the de facto standard for IoT (Internet of Things) communications. While MQTT works seamlessly in local networks, deploying it securely over the Internet requires additional considerations.
In this tutorial, we’ll explore how to implement MQTT over the internet effectively. We’ll cover the fundamental architecture, security considerations, deployment options, and practical implementation examples. By the end, we’ll have a comprehensive understanding of using MQTT in internet-facing applications while maintaining security and reliability.
2. Understanding MQTT Architecture
First, let’s review MQTT’s core architecture to better understand the challenges we may face.
2.1. MQTT Basic Components
MQTT follows a publish-subscribe pattern consisting of three main components:
- MQTT Broker is the central server that receives all messages from publishers and routes them to the appropriate subscribers.
- Publishers are clients sending messages to specific topics on the broker
- Subscribers are clients receiving messages from topics they’ve subscribed to
This architecture decouples publishers from subscribers, allowing for flexible communication patterns. Publishers don’t need to know who receives their messages, and subscribers don’t need to know who publishes them:
2.2. MQTT Topics and Quality of Service
MQTT organizes messages using a hierarchical topic structure (e.g., home/livingroom/temperature). This allows for fine-grained control over message routing.
Additionally, MQTT provides three Quality of Service (QoS) levels:
- QoS 0: At most once delivery (fire and forget)
- QoS 1: At least once delivery (confirmed delivery)
- QoS 2: Exactly once delivery (guaranteed delivery)
When deploying MQTT over the internet, these QoS levels are crucial for managing unreliable connections and ensuring message delivery.
3. Challenges of MQTT Over the Internet
While MQTT works efficiently on local networks, several challenges arise when deploying it over the internet.
When considering security concerns, we must acknowledge that MQTT traffic traversing the public internet is vulnerable to various threats. Unencrypted communications can be intercepted through eavesdropping, while man-in-the-middle attacks may alter traffic in transit. Unauthorized access is another significant risk, as malicious actors can attempt to connect to the broker without proper authentication. Furthermore, denial of service attacks pose a threat when attackers flood the broker with connections or messages, potentially bringing down the entire system.
Network reliability issues present additional hurdles when deploying MQTT over the internet. Intermittent connectivity is common, especially with mobile devices or IoT nodes that frequently disconnect due to changing network conditions. Firewalls and Network Address Translation (NAT) implementations can block MQTT traffic if not properly configured, creating connectivity barriers. Variable latency introduced by internet routing can cause unpredictable delays in message delivery, while limited bandwidth on some internet connections may restrict data rates and affect performance, particularly for devices in remote locations or on congested networks.
Finally, scalability requirements become more demanding for internet-facing MQTT deployments. These systems often handle large numbers of clients, potentially scaling to thousands or even millions of devices connecting simultaneously. Further, high message volumes are another consideration, with many messages per second flowing across numerous topics, creating significant processing and storage demands. Global distribution adds complexity, as clients connecting from various geographic locations may experience different network conditions and latency profiles. Addressing these multifaceted challenges requires a thoughtful combination of proper configuration, robust security measures, and well-designed infrastructure to ensure reliable and secure MQTT communications over the internet.
4. Securing MQTT Over the Internet
Security should be our primary concern when deploying MQTT over the internet. Let’s explore essential security measures.
4.1. Transport Layer Security (TLS/SSL)
Encrypting MQTT traffic is non-negotiable for internet deployment:
# Standard MQTT ports
Port 1883: Unencrypted MQTT (NOT recommended for internet use)
Port 8883: MQTT over TLS (recommended)
To implement TLS with the popular Mosquitto broker, we add these lines to the configuration file:
listener 8883
cafile /etc/mosquitto/ca_certificates/ca.crt
certfile /etc/mosquitto/certs/server.crt
keyfile /etc/mosquitto/certs/server.key
require_certificate false
This configuration:
- enables TLS on port 8883
- specifies the Certificate Authority (CA) file
- defines the server certificate and private key
- makes client certificates optional
We should generate proper SSL certificates using a tool like OpenSSL or obtain them from a recognized Certificate Authority for production use.
4.2. Authentication and Authorization
Even with encryption, controlling who can connect to our broker and what they can access is critical. We can further secure communication with the username/password control in the Mosquitto broker:
allow_anonymous false
password_file /etc/mosquitto/passwd
Then, we can create passwords for users using the mosquitto_passwd tool:
mosquitto_passwd -c /etc/mosquitto/passwd myuser
For higher security, we can require client certificates:
listener 8883
cafile /etc/mosquitto/ca_certificates/ca.crt
certfile /etc/mosquitto/certs/server.crt
keyfile /etc/mosquitto/certs/server.key
require_certificate true
use_identity_as_username true
Finally, we can restrict which clients can publish or subscribe to specific topics using ACLs:
# User-based restrictions
user myuser
topic read home/myuser/#
topic write home/myuser/sensors/#
# Common topics
pattern read notifications/#
4.3. Additional Security Measures
Beyond the basics, we can consider these additional security enhancements:
- Rate limiting to prevent DoS attacks by limiting connection and message rates
- Payload validation, which implements schema validation for message payloads
- IP filtering that restricts connections to known IP ranges when possible
- Regular monitoring for unusual traffic patterns or unauthorized access attempts
5. MQTT Broker Deployment Options
There are several options for deploying MQTT brokers for internet access.
5.1. Self-Hosted Broker
Running our own broker gives us maximum control.
First, we set up a server with a static IP or reliable dynamic DNS, then install and configure an MQTT broker (e.g., Mosquitto, EMQX, or HiveMQ), and finally, we implement all security measures mentioned earlier.
We can use Docker to simplify deployment:
docker run -d
-p 8883:8883
-p 9001:9001
-v /path/to/config:/mosquitto/config
-v /path/to/data:/mosquitto/data
-v /path/to/log:/mosquitto/log
eclipse-mosquitto
This Docker command creates and runs a new container using the Eclipse Mosquitto image. It maps the MQTT over TLS port (8883) and the web interface port (9001) from the container to the host, and mounts three volumes for configuration, data storage, and logging. This makes it easy to deploy an MQTT broker with persistent settings and data while keeping it isolated in a container.
5.2. Cloud-Hosted MQTT Services
Another option is to use one of the many cloud providers that offer managed MQTT services:
- AWS IoT Core: Fully managed MQTT broker with AWS integration
- Azure IoT Hub: Microsoft’s IoT platform with MQTT support
- Google Cloud IoT Core: Google’s managed IoT service
- HiveMQ Cloud: Dedicated MQTT broker service
- CloudMQTT: MQTT as a Service provider
These services handle scaling, security, and reliability, but may have higher costs for large deployments.
6. Handling Disconnections and Reliability on the Client’s Side
Internet connections are inherently unreliable. Let’s see how to make our MQTT implementation robust and reliable in those scenarios.
6.1. Will Messages
MQTT’s “Will” feature sends a message when a client disconnects unexpectedly:
client = mqtt.Client()
client.will_set(
topic="device/status",
payload="offline",
qos=1,
retain=True
)
This allows other clients to be notified of disconnections.
6.2. Persistent Sessions
For subscribers to receive missed messages after reconnecting, we can configure the client to not clean the sessions:
client = mqtt.Client(client_id="unique_client_id", clean_session=False)
Combined with QoS 1 or 2, this ensures messages aren’t lost during disconnections.
6.3. Retained Messages
For new subscribers to immediately receive the latest state, we set up the Retain flag:
client.publish(
topic="device/temperature",
payload="22.5",
qos=1,
retain=True
)
This is especially useful for retaining device state information.
6.4. Reconnection Strategies
Finally, we can implement progressive backoff for reconnection attempts:
def connect_with_retry():
attempt = 0
max_attempts = 12 # Maximum reconnection attempts
while attempt < max_attempts:
try:
delay = min(2 ** attempt, 300) # Exponential backoff, max 5 minutes
print(f"Connection attempt {attempt+1}, waiting {delay}s")
time.sleep(delay)
client.connect(broker, port, keepalive=60)
print("Connected successfully")
return True
except:
attempt += 1
print("Failed to connect after maximum attempts")
return False
This function implements an exponential-backoff reconnection strategy. It makes up to 12 attempts to connect to the broker, with progressively longer wait times between attempts (starting at 1 second and doubling each time, up to the maximum of 5 minutes). This approach prevents overwhelming the broker with rapid reconnection attempts while still ensuring the client will eventually reconnect when the broker becomes available again.
7. Troubleshooting MQTT Over the Internet
Even with careful planning and implementation, we’ll inevitably face issues when running MQTT over the internet. Let’s explore some common problems and how we can methodically address them.
7.1. Connection Problems
Connection issues are probably the most common challenge when deploying MQTT online. When clients can’t connect to the broker, we should first check the basics of network connectivity. Is the broker actually reachable from the client’s network? Firewall configurations are often the culprit here, both on the server and client sides. We must ensure that port 8883 (for TLS connections) is open and properly forwarded if the broker is behind a router.
Certificate issues frequently cause confusing connection problems. We might see clients attempting to connect but immediately disconnecting. In these cases, we should verify that our certificates haven’t expired, that the client trusts the certificate authority that signed the broker’s certificate, and that the hostname in the certificate matches the hostname the client uses to connect. A mismatch in any of these areas will cause TLS validation to fail.
DNS resolution problems can also be tricky to diagnose. Sometimes the domain name we’re using for the broker doesn’t resolve correctly from all client locations. We can test this with a simple ping or dig command to ensure the hostname resolves to the expected IP address.
7.2. Performance Issues
Performance problems typically arise as our MQTT deployment grows. We may notice increased latency, message delivery delays, or broker instability under heavy load.
Message payload size is a key factor affecting performance. We should keep our messages as small as possible, especially for constrained devices or networks. When applicable, consider using efficient encoding formats like Protocol Buffers or CBOR instead of verbose formats like JSON. We might even define custom compact formats that only transmit the essential information for IoT sensors that send similar data repeatedly.
The keep-alive interval setting can significantly impact performance on unstable connections. If we set this value too low, clients frequently disconnect when network conditions degrade. On the other hand, setting it too high means it takes longer to detect when clients have disconnected. Finding the right balance for our specific network conditions is important.
As our deployment scales, monitoring broker resources become essential, so we must watch CPU usage, memory consumption, and the number of concurrent connections. Most MQTT brokers provide metrics endpoints or logging options to help with this. Setting up alerts for abnormal conditions can help us identify problems before they affect users.
8. Conclusion
In this article, we discuss various techniques of deploying MQTT over the Internet, which require careful attention to security, reliability, and network configuration.
By implementing proper encryption, authentication, and connection management, we can create robust IoT systems that communicate effectively across the internet.