Mender 3.6: Auto-generation of delta updates

Mender 3.6 is released, including all the features published on Hosted Mender in the last few months as part of our continuous development and rolling releases.

Mender 3.6 is an LTS (long-term support) release and will be maintained for 12 months from the release date. We encourage all users and customers to upgrade to Mender 3.6 at their convenience.

Automatic bootstrap Artifact and server-side generation of delta Artifacts for operating system updates

Mender Enterprise 3.6 significantly improves support for binary delta operating system updates.

Thanks to the automatic bootstrap Artifact, which is generated at build time in Yocto and automatically installed by the Mender client on the first boot, the correct versioning information is reported by the Mender client from the first boot. The automatic bootstrap Artifact enables operating system updates using binary delta Artifacts from the first update.

Additionally, the server-side generation of delta Artifacts removes the need to generate Binary delta Artifacts for all possible combinations of current and target operating system versions across your fleet of devices. Users can enable this feature on a per-deployment basis by clicking on the corresponding checkbox in the deployment creation screen.

Read more about the server-side generation of Delta Artifacts: Official documentation: Server-side generation of delta artifacts.

CI/CD templates

Mender's API always allowed for simple integration into the CI/CD pipelines. Mender 3.6 takes our integration further, providing ready-to-use production-grade CI/CD pipeline templates of the most common use cases for the major cloud CI/CD providers.

To help users create CI/CD pipelines, a GitHub repository now exists containing ready-to-use, battle-tested CI/CD pipeline code snippets that can be used in your workflows. A mender-ci-tools Docker image based on Alpine Linux is also included, with command-line utilities installed (e.g., mender-artifact and mender-cli).

Mender currently supports three major CI/CD providers:

• GitHub Actions
• GitLab CI/CD
• Azure DevOps

The templates provided include the following actions:

• Creating and signing a Mender Artifact
• Uploading a Mender Artifact
• Deploying an Artifact to your fleet of devices

Read more about CI/CD pipeline support: Official documentation: CI/CD.

Dashboards and reporting

Mender 3.6 includes a revamp of the Dashboard view, which highlights devices with issues, e.g.:

• Offline
• Authentication requests
• Monitoring (if the Monitoring add-on is enabled)

The new and customizable software distribution widgets can be scoped to specific groups, both static and dynamic, reporting operating system versions or specific software applications.

Force installation of Artifacts: Install, even if already installed

The Mender 3.6 server, in combination with the latest version of the Mender client, allows the forced installation of a release. When the “Force update” option is enabled, the server won’t check if the assigned Artifact is already installed and will proceed with the update even if the device is already reporting its installation.

The default behavior (without force) is that Mender will skip the update if it sees the same version is already installed in order to avoid unnecessary downtime and risk. However, during testing (e.g., on your desk) or right after you have manually modified a device (e.g., using the Troubleshoot add-on), it is very useful to be able to force the installation, even though the same version is registered to be installed already.

Using this feature in combination with the –no-default-software-version and –no-default-clears-provides options from mender-artifact creates Artifacts that won’t report software versioning once installed and can be applied multiple times to the same device.

For example, to create an Artifact that runs a script, you could run the following:

echo "exit 0" > script.sh
mender-artifact write module-image -t device-type -T script -n command --no-default-clears-provides --no-default-software-version -f script.sh -o artifact.mender

Deployment history for devices

Imagine you get a support request or notice one of your devices is misbehaving. What makes it different from all the other devices? One key question is: Which software has been installed on this device, and in what order? Knowing this information can help speed up troubleshooting issues dramatically.

Mender 3.6 lists all the deployments for a given device in the new “Deployments” tab within the device details side pane. The deployments in the list are sorted from the most recent to the oldest and can be filtered by status.

Users can also reset the device deployment history; in this case, the entries will appear grayed out and ignored during the deployment rollout. For example, by resetting the deployment history of a device, it is possible to re-trigger an old dynamic deployment for that device, which has already been completed in the past.

Deployment history reset is useful to support devices that are being replaced in the field. If you replace the hardware but keep the same identity for the new device, resetting the device deployment history allows the new device to receive the updates it should, even though the old device had already received them.

Support for the Azure Blob Storage as an Artifact storage layer

Mender now natively supports the Azure Blob Storage Artifact storage layer in addition to the AWS S3-compatible APIs. Native support simplifies the deployment of the Mender Server in Microsoft Azure Cloud Computing environments and related operations.

Check the official Helm chart documentation for the configuration settings required to connect a Mender Server to an Azure Blob Storage account.

Versioning of single-file artifacts

When generating single-file Artifacts using the UI, users can specify the versioning information, including the software file system (e.g., rootfs-image or data-partition), the software name, and the software version.

Deprecation: Synchronized updates

The feature known as Synchronized updates will not be available in the Mender client version 4.0.0, the new Mender client rewritten in C++.

The Synchronized updates feature will continue to be supported on Hosted Mender as long as the Mender client released with the LTS 3.6 release bundle (Mender client 3.5) is supported, currently confirmed until 2024-07.

What did this feature do?

The Synchronized updates feature allowed you to put pauses into certain stages of the update process. The original use case was to allow for the synchronization of the updates on multiple devices. For example, if you had a system dependent on multiple redundant devices of the same type, all of which needed to be updated at the same time, you would set a pause after the download state, wait for all the devices to download the content, and then, trigger the installation on all devices at the same time.

How do I know if I'm using it?

You are using this feature if you are:

1. Setting pauses in the Mender UI.

2. Setting the update_control_map object in any of the API calls.
3. Setting the control map on the device using the DBus API.

Why are we deprecating Synchronized updates?

According to feedback, this feature is showing nearly zero usage in field applications. In addition to usage, this feature is significantly complex to maintain within the current codebase and would result in delaying future releases, such as the C++ client.

I depend on this feature, are there any alternatives planned?

In 2024, we will release an add-on providing the Synchronized updates functionality in a more comprehensive and robust manner, among other features all focused on updating complex systems consisting of interdependent hardware devices. The idea behind the new add-on is to provide synchronization of multiple software versions within the boundaries of the device.

We are currently in the product discovery phase, if you are an Enterprise customer and want to share your use case, please contact us on support@mender.io.

I am using the Synchronized updates feature for different use cases, what should I do?

If you're using synchronized updates in a way different than described here, please let us know. We'd like to understand how you used the feature and look for an alternative together. Please contact us on support@mender.io.

Try the new features

Get started is the best place to do a quick test of the new release from scratch and get started with the new Mender 3.6 features.

All the new features are also available in Hosted Mender, our managed solution. Sign up for the free trial; all features and add-ons are available for 12 months for free - no credit card required.

Share your feedback

We appreciate your general feedback on Mender, be it positive or need for improvement, in the Mender Hub General Discussions forum. Your continued feedback ensures Mender will meet your needs even better in the future!

If you believe you have encountered a bug, please submit your report at the Mender JIRA issue tracker.

We hope you enjoy the new features and look forward to hearing from you!

Update Nov 2, 2023: The section on the deprecation of control maps was mistakenly missed in the original version. This has been updated to reflect the changes in release 3.6 accurately.