Embedded Linux

Metrics with Embedded Linux

Collect Linux OS metrics and report custom application metrics through a local Unix socket.

spotflowd can collect OS metrics from /proc and /sys and accept custom metrics from local applications without an SDK dependency. Both sources use the same aggregation pipeline and MQTT/TLS connection.

Prerequisites

Install and configure the daemon using the Embedded Linux quickstart or Yocto integration. Its state directory must be writable even for metrics-only deployments because metric sequence numbers are persisted there.

Enable OS Metrics

OS metrics are disabled by default. Enable them in /etc/spotflow/spotflowd.toml:

/etc/spotflow/spotflowd.toml
[metrics]
aggregation_interval = "1m"

[metrics.os]
enabled = true
collection_interval_secs = 10

All groups below are enabled when OS collection is enabled. Availability of individual values depends on your kernel and hardware.

GroupMetrics
cpucpu_utilization_percent, cpu_load_avg_1m, cpu_load_avg_5m, cpu_load_avg_15m, cpu_temperature
memorymem_available_bytes, mem_used_percent, swap_used_percent
diskdisk_free_bytes, disk_used_percent, disk_inodes_used_percent, disk_read_bytes, disk_write_bytes, disk_read_ops, disk_write_ops, disk_io_util_percent
networknetwork_rx_bytes, network_tx_bytes, net_rx_errors, net_tx_errors, net_rx_drops, net_tx_drops
systemuptime_ms, process_count, fd_used, fd_max

To reduce collection on constrained devices, disable groups or limit monitored mounts and network interfaces:

/etc/spotflow/spotflowd.toml
[metrics.os.groups]
disk = false
network = true

[metrics.os.disk]
mount_points = ["/"]

[metrics.os.network]
interfaces = ["eth0"]

Replace eth0 with your device's interface name. An empty interfaces list auto-detects non-loopback interfaces. Mount selection applies when the disk group is enabled.

Enable Custom Application Metrics

Enable the Unix domain socket listener:

/etc/spotflow/spotflowd.toml
[metrics.custom]
enabled = true
socket_path = "/run/spotflow/metrics.sock"

Restart the daemon after configuration changes:

>_ Device Terminal
sudo systemctl restart spotflowd
sudo spotflowd status

On Yocto SysVinit devices, use /etc/init.d/spotflowd restart as root instead.

OS and custom metrics have independent enable switches. Custom metrics work with [metrics.os] enabled = false, and enabling OS metrics does not enable the socket.

Report Custom Values

Connect to the socket, write one or more newline-delimited JSON objects, and close the connection. Each object has a name, a numeric value, and optional string-to-string labels:

{"name":"queue_depth","value":42}
{"name":"job_duration_ms","value":183.5,"labels":{"worker":"main"}}

For a shell test using OpenBSD netcat (-N closes the connection after input ends):

>_ Device Terminal
echo '{"name":"queue_depth","value":42}' | nc -UN /run/spotflow/metrics.sock

BusyBox netcat may not support Unix sockets. Applications can write directly using their language's socket API, for example in Python:

metrics.py
import json
import socket

def send_metric(name, value, labels=None):
    message = {"name": name, "value": value}
    if labels:
        message["labels"] = labels
    with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as connection:
        connection.connect("/run/spotflow/metrics.sock")
        connection.sendall(json.dumps(message).encode() + b"\n")

send_metric("queue_depth", 42)
send_metric("job_duration_ms", 183.5, {"worker": "main"})

No response is sent back: writing to the socket is not an acknowledgement of cloud delivery. The socket is local-only but allows any local user to submit values by default; use it only with trusted local processes.

Analyze Metrics in Dashboards

After the aggregation window completes and the device is connected, open the Device Dashboard for device metrics. Use Custom Dashboards to visualize application-specific values and labels.

Collection and Aggregation

collection_interval_secs controls OS sampling frequency. aggregation_interval controls the shared aggregation window for OS and custom metrics:

ValueBehavior
"none"No aggregation window; publish individual samples
"1m"Aggregate over one minute (default)
"1h"Aggregate over one hour
"1d"Aggregate over one day

Aggregated values include sum, count, minimum, and maximum. Longer windows reduce message frequency; shorter collection intervals provide more samples at the cost of local processing. Pending metric values stay in memory, not in the log disk spool, and are lost on daemon exit or power loss. Metrics use best-effort MQTT QoS 0, so local publishing does not confirm receipt by Spotflow.

Reference Repository Materials

Learn more

On this page