Logging with MQTT

Collect logs from any device using standard MQTT interface.

This guide explains how to enable log collection on devices running a MQTT client and analyze logs in the Spotflow web application.

Connect to the Spotflow MQTT broker

As the very first step, check the following basic integration guides:

Publish logs in JSON or CBOR format

Once your client is connected, you can start sending logs. You can choose to send logs in JSON or CBOR format. Prefer CBOR for efficiency in bandwidth-constrained environments. Use JSON for convenience when slightly larger messages are acceptable.

To send logs serialized in JSON, simply publish MQTT messages into the ingest-json topic. The messages should follow the schema below:

{
    // Fully interpolated log line string (optional when bodyTemplate is used)
    "body": "SmartLock was Unlocked",

    // (Optional) printf-like interpolation string for the log line
    "bodyTemplate": "SmartLock was %s",

    // (Optional) array of values for interpolation in the bodyTemplate
    "bodyTemplateValues": ["Unlocked"],

    // (Recommended) Log severity, possible values: "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"
    "severity": "INFO",

    // (Optional) device uptime when the log was generated (milliseconds since the device booted)
    "deviceUptimeMs": 23455,

    // (Optional) time when the log was generated (milliseconds since the UNIX epoch)
    "deviceTimestampMs": 1748530133808,

    // (Optional) you can add extra metadata to your logs
    "labels": {
        "initiatorKind": "MobileApp",
        "userId": "1234567890"
    }
}

Tip for Testing: use a command-line MQTT client to test messages without writing code. Note, that the publish command within these CLI clients typically handles the entire sequence for you: it connects, sends the message, and disconnects.

First, download and install the MQTTX CLI from their website. Then, use the following command to send a log message:

mqttx pub \
--hostname 'mqtt.spotflow.io' \
--port 8883 \
--topic 'ingest-json' \
--protocol 'mqtts' \
--username 'quickstart_device' \
--password '<your-ingest-key>' \
--message '{"body": "Quickstart Device Booted Up", "severity": "INFO", "deviceUptimeMs": 10, "labels": {"location": "facility-123"}}'

Mosquitto ships with a mosquitto_pub command line MQTT client. Make sure you have mosquitto_pub available in your PATH. Then, use the following command to send a log message:

mosquitto_pub \
-h 'mqtt.spotflow.io' \
-p 8883 \
-t 'ingest-json' \
-u 'quickstart_device' \
-P '<your-ingest-key>' \
-m '{"body": "Quickstart Device Booted Up", "severity": "INFO", "deviceUptimeMs": 10, "labels": {"location": "facility-123"}}' \
--capath /etc/ssl/certs

Depending on your operating system and its configuration, you may need to customize the command to point it to a proper location containing the ISRG Root X1 TLS certificate via --capath or --cafile options.

To send logs serialized in CBOR, simply publish MQTT messages into the ingest-cbor topic. The payload should include a single log following the log-message schema as defined below:

log-message = {
    ? 1 => tstr,                  ; body: fully interpolated log line string (optional when bodyTemplate is used)
    ? (
        2 => tstr,                ; bodyTemplate (optional): printf-like interpolation string
        3 => body-template-values ; bodyTemplateValues (optional): values for interpolation
    ),
    ? 4 => severity,              ; severity (recommended)
    ? 5 => labels,                ; labels (optional): user-defined key-value pairs for additional context
    ? 6 => uint,                  ; deviceUptimeMs (optional): device uptime in milliseconds in range [0, 2^63 - 1]
    ? 7 => uint                   ; deviceTimestampMs (optional): device timestamp in milliseconds in range [0, 2^63 - 1]
}

labels = {* (tstr => (tstr / int / float / bool))} ; Strongly typed key-value pairs

; Integer severity values
debug-severity = 30
info-severity = 40
warning-severity = 50
error-severity = 60
critical-severity = 70
severity = (debug-severity / info-severity / warning-severity / error-severity / critical-severity)

; A strongly typed array of values, or an array of byte string representations (big-endian) of the values
body-template-values = [ * (tstr / int / float / bool / null) ] / [ * bstr ]

Analyze Logs in the Web Application

Once your device is integrated and sending logs, you can analyze them in the web application. The main entry point is the Events page, which gives you a comprehensive view of all events collected from your devices. There, you can filter logs by their content, device ID, severity, and other metadata.

Spotflow Web Application Screenshot
The Events page displays all logs with filtering and search capabilities.

You can click on individual log messages to see their details, and it is possible to drill down into specific events by matching their metadata.

Detail a single log message.
It is possible to query logs based on metadata of a particular log message.

Learn more

How is this guide?