Skip to main content

Changelog

Delta Egress can now be managed via Portal and CLI

Spotflow customers can now manage Delta Egress sinks and routes via Portal and spotf CLI.

The newly available CLI commands are:

Creating example Egress Sink
spotf egress-sink create-or-update delta azure-blob \
--name 'delta-sink' \
--connection-string 'AccountName=...;AccountKey=...' \
--container-name 'tables'
Creating example Egress Route
spotf stream egress-route create-or-update delta \
--name 'delta-route' \
--stream-group-name 'example-group' \
--stream-name 'example-stream' \
--egress-sink-name 'delta-sink' \
--directory-path 'table/path'

New tutorials on data routing to observability backends and Azure EventHubs

We've added new tutorials showing how easy it is to send data to Azure EventHubs or observability backends using Spotflow IoT Platform. The tutorials also instruct you how to read the incoming data or visualize them, respectively.

Sounds interesting? Give it a try right away!

Improved on-boarding experience

We've introduced a new interactive quickstart tutorial to improve the first interactions with the platform.

When each user signs up, we setup an initial workspace for them, so there are no obstacles to start using the platform. When the user lands in our Portal, they'll see a tutorial which guides the user through the initial steps within the platform:

  1. Using the Device SDK.
  2. Provisioning a new device.
  3. Sending messages into the platform.
  4. Visualizing the data in our built-in Grafana.

Device Fleet Configuration templates are now fully editable

Spotflow customers using Device Fleet Configurations to configure devices at scale can now modify the configuration's content or path.

This feature allows to edit configuration used by many devices without needing to delete and create a new configuration. Our goal with this update is to make it easy for customers to make iterative changes to large scale of devices even when running in production.

Please see Tutorial: Configure Multiple Devices if you want to start using Device Fleet Configurations.

Documentation overhaul: Now more user-friendly and use-case oriented!

We've completely redesigned our documentation to be more intuitive and focused on real-world use cases. Dive in to find clearer guides and more helpful examples tailored just for you!

Devices can now be deleted via CLI

Spotflow users can now delete devices directly through the command line interface (CLI). This feature provides a quick and efficient way to delete devices without the need to access the portal. To delete a device, you can use the spotf device delete command followed by the device's ID. Here's an example:

spotf device delete --device-id "<device_id>"

Please refer to the CLI command reference documentation for detailed instructions.

Route data AWS S3

AWS S3 Egress

Spotflow customers can now seamlessly route data streams from devices into AWS S3 buckets. This feature is provided as a new Egress Sink kind to which both new and existing streams can be routed.

See detailed documentation in Amazon S3 Egress Sink page.

The Target S3 bucket is specified via bucket name, region, and IAM user access key. Optionally, a static prefix of the target path can be added. The new egress sink can be configured via Portal, CLI, or API.

Previously, this use case was possible only by writing custom processors consuming data through one of the existing egress sinks, such as Azure Event Hub. This option is still viable for existing and new customers, but we recommend migrating to the new AWS S3 egress sink.

Creating Egress Sink
Creating Egress Route

Route data to Delta Tables in Azure Blob Storage

Delta Egress

Data streams from devices can now be directly inserted into Delta Tables backed by Azure Blob Storage. This feature is provided as a new Egress Sink kind to which both new and existing streams can be routed.

The target Azure Blob Storage container is specified via connection string and container name. The Delta Table is created in the specified container, and the data is inserted into it. Optionally, a static prefix of the target path can be added.

The created Delta Table contains several predefined columns, such as stream_name, message_id or ingress_enqueued_date_time as well column payload_line with content of the stream message.

See detailed documentation in Delta Egress Sink page.

The new egress sink can be configured via API. Support in CLI and Portal is not yet available.

Creating egress Sink
#!/usr/bin/env pwsh

$body = @{
properties = @{
config = @{
delta = @{
azureBlobStorage = @{
connectionString = "AccountName=...;AccountKey=..."
containerName = "delta"
}
}
}
}
}

$uri = "https://api.eu1.spotflow.io"
$uri += "/workspaces/$workspaceId"
$uri += "/egress-sinks/delta-example"

Invoke-WebRequest `
-Method Patch `
-Uri $uri `
-Headers @{ "Authorization" = "Bearer $token" } `
-ContentType "application/json" `
-Body ($body | ConvertTo-Json -Depth 10)
Creating egress route
#!/usr/bin/env pwsh

$body = @{
properties = @{
egressSinkName = "delta-sink-sample"
config = @{
delta = @{
directoryPath = "sample-path"
}
}
}
}

$uri = "https://api.eu1.spotflow.io"
$uri += "/workspaces/$workspaceId"
$uri += "/stream-groups/$streamGroupName"
$uri += "/streams/$streamName"
$uri += "/egress-routes/delta-route-sample"

Invoke-WebRequest `
-Method Patch `
-Uri $uri `
-Headers @{ "Authorization" = "Bearer $token" } `
-ContentType "application/json" `
-Body ($body | ConvertTo-Json -Depth 10)

Product changelog started

Product changelog has been started. Spotflow engineers and product managers will keep you informed about the latest updates and improvements via this page. Stay tuned!