Using Spotflow in Mesh Networks

Many embedded deployments use networks where only one device has internet connectivity. Sensor nodes, actuators, and other leaf devices communicate over Bluetooth Low Energy, Zigbee, proprietary radio, or wired buses, and rely on a single gateway to reach the cloud. This page explains how to collect logs, metrics, and crash reports from those leaf devices in Spotflow, what you must build yourself, and what the platform handles for you.

Architecture

The central idea is straightforward: leaf devices serialize Spotflow messages and pass them over the local link toward a gateway device, optionally through one or more relay nodes. Each relay node forwards raw message bytes over the next local link hop without inspecting or modifying the content. The gateway forwards each message to the Spotflow MQTT broker under its own connection. Spotflow attributes the record to the originating leaf device, and the gateway's ID appears as the transport route.

Mesh network architecture showing leaf devices (LD1, LD2) communicating via local link (BLE, Zigbee, etc.), optionally through relay nodes (RN1), with the gateway (GD1) forwarding all messages to Spotflow via MQTT.

Spotflow automatically registers every device ID forwarded by a trusted, authenticated gateway, so no pre-registration of leaf devices is required. The gateway authenticates to Spotflow using its ingest key; leaf device identities are accepted transitively through that trust. Once the first relayed message arrives, the leaf device appears in the portal with its own device page, event history, and dashboards. The transport route (the full chain of device IDs from origin to cloud) is recorded on each event.

How Spotflow Handles Relayed Messages

The mechanism that makes this work is the sourceDeviceId field. Every Spotflow message format (logs, metrics, core dump chunks) accepts this optional field. When present, the cloud uses it as the authoritative device ID for that record. The device that actually connected to MQTT (the gateway) is recorded in the transport route instead.

The resolved device ID is always the original source. This is the ID you will see in the portal, in queries, and in dashboards.

Constraint: Each device ID must consist of alphanumeric characters, hyphens, underscores, dots, and colons; maximum 64 characters.

Adding sourceDeviceId to JSON messages

For any message type published to the ingest-json topic, add the sourceDeviceId field at the top level:

{
  "body": "Sensor node booted",
  "severity": "INFO",
  "deviceUptimeMs": 500,
  "sourceDeviceId": "ld1"
}

{
  "messageType": "METRIC",
  "metricName": "temperature_celsius",
  "sum": 23.5,
  "deviceUptimeMs": 60000,
  "sequenceNumber": 1,
  "sourceDeviceId": "ld1"
}

{
  "messageType": "CORE_DUMP_CHUNK",
  "coreDumpId": 987654321,
  "chunkOrdinal": 0,
  "content": "WkUCAAMABQAD...",
  "isLastChunk": false,
  "buildId": "build-ld1-v1.2.0",
  "os": "Zephyr",
  "sourceDeviceId": "ld1"
}

Adding sourceDeviceId to CBOR messages

For CBOR-encoded messages published to the ingest-cbor topic, sourceDeviceId uses property key 31:

; CBOR field reference
31 => tstr                ; sourceDeviceId: originating leaf device ID

This applies to all three message types (log messages, metric messages, and core dump chunks). The field is appended to the existing CBOR map alongside the message-type-specific fields.

Current Limitations

The sections below describe exactly how to implement this relay logic in your application firmware.

The Gateway Device

The gateway is the only device in the network with an MQTT connection to Spotflow. It connects using its own ingest key with its device ID as the MQTT username, exactly as a directly connected device would. See the MQTT quickstart for setup.

The gateway's job is to:

1. Receive raw message bytes from leaf devices over the local link.
2. Reconstruct the complete message from the local link.
3. Publish the bytes as-is to the appropriate MQTT topic (ingest-json or ingest-cbor).

The gateway treats each message as opaque bytes, it does not need to inspect or modify the content, because sourceDeviceId is already embedded by the leaf device. The gateway also collects its own telemetry using the Spotflow device module normally. Its logs, metrics, and core dumps are sent directly without a sourceDeviceId field.

The Relay Node

A relay node is an intermediate device that sits between one or more leaf devices and the gateway. It does not connect to Spotflow directly and does not need an ingest key. Its only responsibility is to forward raw message bytes from leaf devices toward the gateway over the next local link hop.

The relay node's job is to:

1. Receive raw message bytes from leaf devices over the local link.
2. Forward the bytes as-is to the next hop (another relay node or the gateway).

Because sourceDeviceId is already embedded in the message by the originating leaf device, the relay node does not need to inspect or modify the content. Multiple relay hops are supported; the bytes pass through each hop unchanged until they reach the gateway.

A relay node may also act as a leaf device and send its own telemetry to the gateway, in which case it follows the same rules as any other leaf device.

The Leaf Device

The leaf device's job is to produce serialized Spotflow messages and deliver them to the gateway. For each message:

1. Serialize the message payload in CBOR (preferred for bandwidth efficiency) or JSON, including sourceDeviceId set to the leaf device's own device ID.
2. Transmit to the next hop (a relay node or the gateway directly).

The leaf device does not connect to MQTT and does not need an ingest key. Its device ID in sourceDeviceId is the identifier that will appear in the Spotflow portal.

Device ID consistency

The device ID used in sourceDeviceId must be stable across reboots and consistent across all message types. Use a value derived from hardware (a MAC address, chip serial number, or provisioned identifier) rather than a randomly generated value. If the device ID changes, the portal will treat the device as a new entity.

Message Priority and Queuing

When multiple message types are pending, deliver them in this order:

1. Core dump chunks: highest priority; deliver before anything else.
2. Metrics: second priority; deliver when no core dump is in progress.
3. Logs: lowest priority; deliver when neither core dumps nor metrics are pending.

This matches the priority order used internally by the Spotflow device module and ensures that crash data (the most diagnostically critical) reaches the cloud before routine telemetry.

For local link transports that support different reliability modes (acknowledged vs. unacknowledged), consider sending core dump chunks as acknowledged messages and logs as unacknowledged. This trades throughput for crash data integrity.

Core Dump Reliability

A core dump is split into numbered chunks, all sharing the same coreDumpId. Spotflow requires all chunks to arrive (though not necessarily in order) to reconstruct and analyze the dump. Missing chunks result in an incomplete crash report.

To maximize reliability:

• Use the acknowledged (reliable) mode of your local transport for core dump chunk transfers where available.
• Store undelivered core dump chunks in non-volatile storage on the leaf device. Core dump data should survive a reboot so it can be retransmitted if the gateway connection was not available at crash time.
• Track delivery progress per chunk on the leaf device. Mark each chunk as delivered only after the gateway confirms receipt at the local link level.

Network Topologies

Any topology where one or more gateways forward leaf device messages to Spotflow is supported, device IDs are stable regardless of which gateway forwards a given message. Relay nodes may be inserted between leaf devices and the gateway to extend range or bridge network segments; messages pass through each relay hop unchanged. The one constraint is that each leaf device must route all its messages through a single gateway. If the same message reaches Spotflow via two different gateways, it will appear as a duplicate in the portal. Ensuring single-gateway routing per leaf device is the responsibility of your network and application code.

Learn More