Top Trend #2: Automation Enables Scale for Fleet Management

Image inspired by upklyak on Freepik

Over-the-air software updates bring automation at scale that is essential for a modern connected device fleet

If you start up an embedded project and have less than 10 connected devices in one location that require system and application updates, then it would be relatively easy to update them manually with a USB key. It might get a little bit harder if each of these devices were distributed across the planet with one in New York, another in Berlin, another in Timbuktu, one in Kuala Lumpur, one in Shanghai, one in Sao Paulo, one in Cape Town, one in San Francisco, one in Santiago; and finally one way down under in Sydney. Then you would have to either fly to each of these exotic locations to update them (I jest) or entrust local engineers to go on site in each location to update each of these devices with a USB key. This then becomes an uncertain social process: How would you know if the right software version has been updated on these devices and what if you had a requirement for synchronized updates across all these devices? How would you ensure that the engineers performed the local updates at exactly the same time? The risk and costs rise pretty quickly in this scenario.

The above scenario explains to you why it is a no-brainer to invest in an OTA update solution with automation capabilities if you have plans to scale a fleet of devices to hundreds and thousands of devices across remote locations. In this blog article, we’ll consider the scenarios where automation in OTA, typically found in a solution like Mender enterprise can be employed to enable increased operational efficiency, cost reduction, ensure security, decrease risk, ensure compliance and increase business value.

Integration into enterprise systems

Typically, large enterprises manage the applications and telemetry data for their connected devices in a cloud IoT platform such as AWS IoT Core or Azure IoT Hub. Mender comes into play when these connected devices need both system and application updates and would benefit from a “device-centric” web UI and integrated management service and client for managing the updates that are deployed across the fleet. Mender as an OTA solution has reference integrations with both Azure IoT Hub or AWS IoT Core so it can be used as a front door to the device fleets in these services. Take AWS IoT Core as an example: Mender as the advanced OTA updater could automatically accept a device and create a device in AWS IoT Core, Mender can then provision the AWS IoT device certificate and key, authenticate and check for updates. This reduces burden for the fleet manager / operator as just one system needs to be used to manage the devices yet data can be synchronized across the two platforms and reused in other enterprise applications. Webhooks can also be used to easily integrate Mender as an OTA update manager with other 3rd party platforms in an enterprise ecosystem. The web hooks integration will notify other systems automatically about device lifecycle events and reduce operational burden by avoiding polling and manual synchronization between the OTA updater and other systems.

Cost reduction

Connected devices may be in remote locations and these devices may depend on cellular or satellite internet connections: Here having an automated delta updates capability in the OTA tool is strategically essential. Deploying a full system software image to these devices will be bandwidth intensive and so a data reduction method is needed to deploy this software image at scale to many hundreds or thousands of devices and one that will adapt to the cellular or satellite connection. On average, a full rootfs software image update can be 269 Mbyte in size. When it is compressed to the delta for an update, it shrinks to just 30 Mbyte in size. The Mender product engineering team have calculated that the average cost savings with a delta update for one device and one update is 89%. With deployments of 1,000 devices and 4 updates per year this can amount to cellular costs of $1,484 with delta updates and $13,310 with full image updates (no delta). That is a considerable saving for the business and one that becomes dramatically larger as the fleet size increases and your connected product becomes more successful.

Consistent software versioning and bulletproof security

The best examples of consistent software versioning and ensuring bulletproof security with OTA software updates is the workflow of zero touch provisioning. This is where a fleet of connected devices - let’s say IoT gateways on domestic boilers - is provisioned in the factory and deployed into the field. The first time the devices are connected, they automatically poll the back end server securely for the latest full system update. Regardless of whether the devices have been sitting in a warehouse for 12 months or even longer than that, if they have the correct identity, they will be verified by the OTA server and get the latest software version as soon as they are installed and connected. The edge gateway has a secure update process from factory to the field. Certificates provided by a certificate provisioning service such as Edgelock2GO authenticate devices to the cloud services; Mender’s mutualTLS ambassador is integrated with a secure element (HSM) in the device hardware so that from factory provisioning, the pre authorized device could use the certificate and the secure element via an OpenSSL engine to connect securely to all of backend services and poll for updates. Dynamic deployments, A/B partition and roll back are further automations that minimize operational burden and optimize device robustness.

Safety, risk reduction and compliance

In certain industries, software deployments need to be scheduled and carried out in dedicated maintenance windows. Mender allows for the automatic scheduling of deployments so updates can be deployed when a device is idle. This saves on the support cost of manually triggering deployments. In some cases the update needs to be synchronized across the fleet within that window. Mender does this in updating aircraft cabin modules with Airbus. The synchronized updates enable shorter maintenance windows, provide user-controlled downtime and help ensure the quality of installed software.

Furthermore, the software update may need to be verified by authorized personnel before the artifact is deployed to the devices. This requires role-based access control (RBAC) and synchronized updates to work in the OTA update solution so that a certified maintenance employee can activate a certain software update and then document/validate the change.

Phased rollout in the OTA solution enables a gradual roll out of an update and this reduces the risk of fleet malfunction and lowers costs due to unforeseen issues.

Increase business value

OTA helps to achieve faster time to market for a connected product. For example, automatic retry of failed deployments can reduce update cycle times and enables rapid feature and service introduction. ZF got its product out to market faster by being able to test and quality assure features after its GRT Shuttle Autonomous Guided Vehicle (AGV) was placed in the field. Framery used OTA updates to release fast and take customer feedback from a first generation connected office pod product and then added new features under subscription OTA. For example, they thought a noise canceling capability would be the most attractive feature, but then found out that a pod meeting scheduling application accessible by end user smartphone would be very good to offer as an upgrade service under subscription. They enabled this through OTA.

Test Mender out and you can scale at your project’s pace

Try Mender on a Raspberry Pi device and the full feature set on a trial for up to 10 devices. It will support all your automation needs if you decide to scale your fleet quickly regardless of whether you stay on the Raspberry Pi / Debian platform or port to a more streamlined embedded Linux (Yocto) as you move into full production.