Logging

Spotflow provides a logging solution designed specifically for embedded devices.

This guide walks you through the complete logging workflow: from integrating the logging module into your device firmware, through understanding the transport protocol, to analyzing and troubleshooting logs in the web application.

Getting Started with Device Integration

Spotflow offers native integration for devices running Zephyr and Nordic nRF Connect SDK through a lightweight software module. This module integrates seamlessly with your existing logging infrastructure - simply add it as a dependency and use the standard logging macros you're already familiar with.

For devices running other platforms or when you need custom integration, you can implement logging using our MQTT-based protocol.

See the guides below for specific integration instructions:

How the Device Module Works

The Spotflow device module integrates with Zephyr's native logging subsystem to provide a familiar development experience. Instead of replacing your existing logging workflow, it extends it by implementing a custom logging backend that captures logs from standard Zephyr macros and transmits them to Spotflow.

Diagram of the device module architecture.

Key benefits of this architecture:

Zero code changes: Continue using LOG_DBG(), LOG_INF(), LOG_WRN(), and LOG_ERR() macros as before
Local debugging: Our device module can run alongside other logging backends, allowing you to see logs in your local console while also transmitting them to Spotflow.

Example: Adding Logs to Your Application

#include <zephyr/logging/log.h>

LOG_MODULE_REGISTER(my_app, LOG_LEVEL_DBG);

void sensor_reading_handler(int temperature) {
    if (temperature > 85) {
        LOG_ERR("Temperature critical: %d°C", temperature);
    } else if (temperature > 70) {
        LOG_WRN("Temperature elevated: %d°C", temperature);
    } else {
        LOG_INF("Temperature normal: %d°C", temperature);
    }
}

These logs will be consumed by Spotflow and your existing logging backends (such as the built-in UART backend from Zephyr), with no additional code required.

Handling Network Interruptions

The device module is designed to handle network interruptions gracefully. When the connection is unavailable, logs are stored in a circular buffer. That is, when the buffer is full, the oldest logs are overwritten with the new ones. Once the connection is restored, buffered logs are automatically sent to Spotflow.

Support for Constrained Devices

To minimize the memory footprint, the device module exposes number of configuration options to tune the buffer size and other parameters. See the available Kconfig options for details.

Transport Protocol

Spotflow uses an optimized transport protocol designed for embedded devices with limited bandwidth and processing power.

MQTT over TLS

All log transmission uses MQTT over TLS (MQTTS) for secure, reliable communication:

TLS Version: 1.2 or higher
Certificate: Let's Encrypt ISRG Root X1 (pre-installed on most systems)
Authentication: Devices authenticate using ingest keys as MQTT passwords and their unique device IDs as MQTT usernames.

Log Message Format

Spotflow uses CBOR (Concise Binary Object Representation) for log serialization to minimize bandwidth usage in constrained environments. All keys and static values use single-byte identifiers to reduce overhead.

The log messages follow the following CDDL schema:

log-message = {
    ? 1 => tstr,                  ; body: fully interpolated log line string (optional when bodyTemplate is used)
    ? (
        2 => tstr,                ; bodyTemplate (optional): printf-like interpolation string
        3 => body-template-values ; bodyTemplateValues (optional): values for interpolation
    ),
    ? 4 => severity,              ; severity (recommended)
    ? 5 => labels,                ; labels (optional): user-defined key-value pairs for additional context
    ? 6 => uint,                  ; deviceUptimeMs (optional): device uptime in milliseconds in range [0, 2^63 - 1]
    ? 7 => uint                   ; deviceTimestampMs (optional): device timestamp in milliseconds in range [0, 2^63 - 1]
}

labels = {* (tstr => (tstr / int / float / bool))} ; Strongly typed key-value pairs

; Integer severity values
debug-severity = 30
info-severity = 40
warning-severity = 50
error-severity = 60
critical-severity = 70
severity = (debug-severity / info-severity / warning-severity / error-severity / critical-severity)

; A strongly typed array of values, or an array of byte string representations (big-endian) of the values
body-template-values = [ * (tstr / int / float / bool / null) ] / [ * bstr ]

For custom MQTT implementations, you may use JSON as the serialization format when higher bandwidth is acceptable. See the MQTT Integration Guide for details.

Analyze Logs in the Web Application

Once your device is integrated and sending logs, you can analyze them in the web application. The main entry point is the Events page, which gives you a comprehensive view of all events collected from your devices. There, you can filter logs by their content, device ID, severity, and other metadata.

The Events page displays all logs with filtering and search capabilities.

You can click on individual log messages to see their details, and it is possible to drill down into specific events by matching their metadata.

It is possible to query logs based on metadata of a particular log message.

Logging

On this page