Crash reports with MQTT
Send crash reports and collect core dumps from any device in the field that can run a MQTT client and analyze them with Spotflow.
This guide explains how to send crash reports and core dumps from devices running running a MQTT client and analyze crashes and core dumps in the Spotflow web application.
Connect to the Spotflow MQTT broker
As the very first step, check the following basic integration guides:
Publish core dump in JSON or CBOR format
Core dump first needs to be split into chunks (of up to 900 KiB each) and then each chunk can be sent as a separate MQTT message. Order in which the messages are sent is not critical as long as all chunks have proper ordinal and the last chunk is marked accordingly.
All chunks belonging must have the same core dump ID set which is a non-negative integer with range between 0 and 2^63-1 (LONG_LONG_MAX).
This ID needs to be unique in the scope of a one device (device ID) and last 7 days, not globally.
This implies that senders/devices can generate random integers in a stateless manner for this purposes.
To enable core dump analysis, specify operating system and build ID in the core dump metadata.
To send core dump chunks in JSON, simply publish MQTT messages into the ingest-json topic.
The messages should follow the schema below:
{
// Must be set to CORE_DUMP_CHUNK.
"messageType": "CORE_DUMP_CHUNK",
// Identifier of the core dump unique within a scope of device and last 7-days.
"coreDumpId": 123,
// Zero-based index of the chunk in the sequence.
"chunkOrdinal": 1,
// Base64-encoded chunk data.
"content": "WkUCAAMABQADAAAAQQIARAADAAAAAAAAAElTKgAAAAAAuMIFE...",
// (Optional) Flag indicating if this is the last chunk.
"isLastChunk": false,
// (Optional) Identifier of the ELF file build for linking with symbols.
"buildId": "build-123",
// (Optional) Operating system indicator, currently only "Zephyr" or empty one is supported.
"os": "Zephyr"
}Tip for Testing: use a command-line MQTT client to test messages without writing code.
Note, that the publish command within these CLI clients typically handles the entire sequence for you: it connects, sends the message, and disconnects.
See examples in Quickstart.
To send core dump chunks serialized in CBOR, simply publish MQTT messages into the ingest-cbor topic.
The payload should include a chunk following the core-dump-chunk schema as defined below:
core-dump-chunk = {
0 => 2, ; messageType: must be set to 2 = "CORE_DUMP_CHUNK".
9 => uint, ; coreDumpId: identifier of the core dump unique within a scope of device and last 7-days.
10 => uint, ; chunkOrdinal: zero-based index of the chunk in the sequence.
11 => bstr, ; content: chunk data.
? 12 => bool, ; isLastChunk (optional): flag indicating if this is the last chunk.
? 14 => tstr, ; buildId (optional): identifier of the ELF file build for linking with symbols.
? 15 => tstr ; os (optional): operating system indicator, currently only "Zephyr" or empty one is supported.
}Analyze Crash Reports in the Web Application
Once your device is integrated, you can analyze crash reports and core dumps in the web application.
Each sent core dump automatically generates a crash report.
The main entry point is the Events page, which gives you a comprehensive view of all events collected from your devices. There, you can filter crash reports by their content, device ID and other metadata. Diagnose the crash by analyzing:
- Stack traces of one or more threads.
- Register values and local variables for individual stack frames. If a register value is available, it can be also casted to several types.
- State of global variables at the time of the crash.
Learn more
How is this guide?