Metrics with ESP-IDF

Prerequisites

Before you start integrating your device with Spotflow, make sure you:

• have a development environment with ESP-IDF 5.0+ installed,
• have a device configuration that can connect to the internet,
• have a Spotflow ingest key.

Alternatively, follow the Quickstart: ESP-IDF Integration Guide to integrate Spotflow using a sample application.

Install Spotflow Device Module

Add the spotflow module as a dependency to your idf_component.yml file.

dependencies:
    spotflow:
        git: https://github.com/spotflow-io/device-sdk.git
        path: esp_idf/spotflow
        version: main

Enable metrics in Kconfig

Configure the Spotflow device identity and enable metrics in sdkconfig.defaults:

CONFIG_SPOTFLOW_DEVICE_ID="esp-device-001" # Set unique identifier of your device
CONFIG_SPOTFLOW_INGEST_KEY="{your-ingest-key}" # Set your Spotflow ingest key

# Enable Spotflow metrics collection
CONFIG_SPOTFLOW_METRICS=y

The CONFIG_SPOTFLOW_INGEST_KEY is a secret key that allows your device to authenticate with Spotflow. You can manage your ingest keys on the ingest keys page.

Initialize Spotflow

Include spotflow.h and call spotflow_init() during application startup. When metrics are enabled, spotflow_init() initializes the metrics registry, metrics transport, heartbeat, system metrics, and the Spotflow MQTT client.

Metrics can be reported before MQTT is connected. The SDK queues encoded metric messages locally and sends them once the MQTT connection is available.

#include "spotflow.h"

void app_main(void)
{
    // Initialize required platform services such as NVS and networking setup.

    spotflow_init();
}

Enable system metrics auto-collection

To collect system telemetry automatically, enable:

CONFIG_SPOTFLOW_METRICS_SYSTEM=y

The following system metrics are collected:

By default, stack metrics are collected for all FreeRTOS tasks (CONFIG_SPOTFLOW_METRICS_SYSTEM_STACK_ALL_THREADS=y). If you need to limit this, disable that option and register only selected tasks with spotflow_metrics_system_enable_thread_stack(...).

Analyze system metrics in dashboards

After enabling system metrics, open the Device Dashboard to inspect device vitals, resource usage, and connectivity trends:

• Device Dashboard

The Connectivity & Traffic section of the Device Dashboard.

Report custom metrics

Custom metrics let you track application-specific values such as sensor readings, request latencies, or business counters. The SDK handles aggregation, encoding, and transmission to Spotflow automatically.

Include the metrics header

Include the metrics API header in your application source file:

#include "metrics/spotflow_metrics_backend.h"

This header provides functions for registering and reporting metrics. It transitively includes the type definitions and registration API.

Register metrics

Before reporting values, each metric must be registered with a name and an aggregation interval. Registration returns a handle that is used for all subsequent reports.

Metrics can be either integer (int64_t) or float (float), and optionally support labels for dimensional breakdowns.

Label-less metrics

A label-less metric tracks a single time series:

static struct spotflow_metric_int *counter_metric;
static struct spotflow_metric_float *temperature_metric;

int rc;

/* Integer metric aggregated over 1 minute */
rc = spotflow_register_metric_int(
    "app_counter",
    SPOTFLOW_AGG_INTERVAL_1MIN,
    &counter_metric);

/* Float metric with no aggregation (each value sent immediately) */
rc = spotflow_register_metric_float(
    "temperature_celsius",
    SPOTFLOW_AGG_INTERVAL_NONE,
    &temperature_metric);

Labeled metrics

A labeled metric tracks multiple time series distinguished by label key-value pairs. At registration, specify the maximum number of unique label combinations (max_timeseries) and the maximum number of labels per report (max_labels):

static struct spotflow_metric_float *request_duration_metric;

rc = spotflow_register_metric_float_with_labels(
    "http_request_duration_ms",
    SPOTFLOW_AGG_INTERVAL_1MIN,
    18,  /* max_timeseries: e.g. 3 endpoints x 2 methods x 3 statuses */
    3,   /* max_labels */
    &request_duration_metric);

Each unique combination of label values is tracked as a separate time series with its own aggregation state.

Aggregation intervals

The aggregation interval controls how long values are accumulated before being sent:

Report metric values

After registration, report values from your application tasks.

Label-less values

/* Report an integer value */
spotflow_report_metric_int(counter_metric, 42);

/* Report a float value */
spotflow_report_metric_float(temperature_metric, 23.5f);

Labeled values

Attach labels to each report using an array of struct spotflow_label:

struct spotflow_label labels[] = {
    { .key = "endpoint", .value = "/api/users" },
    { .key = "method",   .value = "GET" },
    { .key = "status",   .value = "200" }
};

spotflow_report_metric_float_with_labels(
    request_duration_metric, 120.5f, labels, 3);

Label keys are stored in 16-byte buffers and values in 32-byte buffers, including the null terminator. Longer keys or values are truncated by the SDK.

Events

For point-in-time occurrences where you only need to record that something happened, use the event API. An event is equivalent to reporting an integer value of 1 with no aggregation:

static struct spotflow_metric_int *button_pressed_metric;

/* Register with no aggregation */
spotflow_register_metric_int(
    "button_pressed", SPOTFLOW_AGG_INTERVAL_NONE,
    &button_pressed_metric);

/* Report that the event occurred */
spotflow_report_event(button_pressed_metric);

Events also support labels:

static struct spotflow_metric_int *app_error_metric;

spotflow_register_metric_int_with_labels(
    "app_error", SPOTFLOW_AGG_INTERVAL_NONE,
    10,  /* max_timeseries */
    1,   /* max_labels */
    &app_error_metric);

struct spotflow_label error_labels[] = {
    { .key = "code", .value = "timeout" }
};

spotflow_report_event_with_labels(app_error_metric, error_labels, 1);

Custom metrics can be visualized in Custom Dashboards.

Analyze custom metrics in dashboards

After reporting custom metrics, open Custom Dashboards to visualize and analyze your application-specific data:

• Custom Dashboards

Custom dashboard example.

Try the metrics example

The SDK includes a complete ESP-IDF metrics example that enables system metrics and reports custom application metrics.

To create a project from the example, run:

idf.py create-project-from-example "spotflow/device_sdk:metrics"

Configure Wi-Fi and Spotflow credentials in sdkconfig.defaults or through idf.py menuconfig, then build and flash the project.

Tune collection and aggregation

System metrics

Use these options to control sampling cadence and the aggregation window before upload:

CONFIG_SPOTFLOW_METRICS_SYSTEM_COLLECTION_INTERVAL=10
CONFIG_SPOTFLOW_METRICS_SYSTEM_AGGREGATION_INTERVAL=60

• CONFIG_SPOTFLOW_METRICS_SYSTEM_COLLECTION_INTERVAL defines how often system metrics are sampled, in seconds.
• CONFIG_SPOTFLOW_METRICS_SYSTEM_AGGREGATION_INTERVAL defines how long samples are aggregated before sending.
• Aggregation interval supports 0 (no aggregation), 60 (1 minute), 3600 (1 hour), and 86400 (1 day).
• With aggregation interval 0, each sample is sent immediately.
• Lower collection interval gives finer time resolution but increases local processing overhead.
• Higher aggregation interval reduces message frequency and network traffic by combining more samples.

Custom metrics

The following Kconfig options control buffering, encoding, and metric limits:

CONFIG_SPOTFLOW_METRICS_QUEUE_SIZE=16
CONFIG_SPOTFLOW_METRICS_CBOR_BUFFER_SIZE=512
CONFIG_SPOTFLOW_METRICS_MAX_REGISTERED=32
CONFIG_SPOTFLOW_METRICS_MAX_LABELS_PER_METRIC=4

• CONFIG_SPOTFLOW_METRICS_QUEUE_SIZE controls how many encoded messages are buffered before MQTT transmission. The default is 16, or 64 when system metrics are also enabled.
• CONFIG_SPOTFLOW_METRICS_CBOR_BUFFER_SIZE sets the buffer size used for CBOR encoding.
• CONFIG_SPOTFLOW_METRICS_MAX_REGISTERED limits how many metrics can be registered simultaneously. System and custom metrics share this pool.
• CONFIG_SPOTFLOW_METRICS_MAX_LABELS_PER_METRIC limits how many label key-value pairs can be attached to a single report.

When MQTT is not connected, metric messages remain in the metrics queue until they can be sent. If the queue becomes full, the SDK drops the oldest queued metric and enqueues the newest one. Tune CONFIG_SPOTFLOW_METRICS_QUEUE_SIZE for expected offline periods and reporting frequency.

Heartbeat

Heartbeat is enabled by default when metrics are enabled. It reports uptime_ms periodically and has higher priority than regular metrics.

CONFIG_SPOTFLOW_METRICS_HEARTBEAT=y
CONFIG_SPOTFLOW_METRICS_HEARTBEAT_INTERVAL=60