How to leverage over-the-air (OTA) updates with NVIDIA Microservices for Jetson

NVIDIA Metropolis Microservices for Jetson is a framework enabling the rapid development of AI applications for IoT products. It offers pre-built microservices and AI models for quickly building vision AI applications for IoT on Jetson. Including generative AI, such as natural language video search and querying, NVIDIA Metropolis Microservices for Jetson significantly shortens the time to market for new applications and products.

Showcased at NVIDIA GTC, a global AI conference running from March 18-21 in San Jose, Calif., NVIDIA Microservices for Jetson is a collection of cloud-native building blocks that enable the rapid development of AI applications for IoT products. They distribute the power of AI, expanding its accessibility to a large audience of developers on NVIDIA Jetson and facilitating the rapid development of new innovative applications by using out-of-the-box solutions like object pattern recognition.

While advantageous, NVIDIA Metropolis Microservices for Jetson is still another software stack that needs to be installed, updated, managed, and supported across the Jetson device fleet, at scale. Leveraging this framework will still encounter challenges in managing large fleets of NVIDIA Jetson modules and its applications once deployed and in production. For example, AI models and applications must be continuously refined and updated to improve their associated products while ensuring the highest levels of security.

To solve these problems, Mender, in collaboration with NVIDIA, published two critical use cases for NVIDIA developers. These resources provide a step-by-step breakdown to leverage over-the-air (OTA) updates within the NVIDIA Jetson environment.

Use case #1: How to install Metropolis Microservices for Jetson stack on Jetson devices

The first use case is a step-by-step guide for developers deploying Metropolis Microservices for Jetson on Jetson devices using Mender. The example requires a Nvidia Jetson AGX Orin devkit or Nvidia Jetson Orin NX 16GB devkit (self built) with 128GB(min) NVMe drive.

Using an NVIDIA Jetson Orin, follow the tutorial, Provisioning "Metropolis Microservices for Jetson" to Nvidia Jetson devices by using Mender, to quickly bring up Metropolis Microservices for Jetson - even for those already in the field.

Use case #2: How to deploy AI applications that use Metropolis Microservices for Jetson

The second use case covers how to deploy a new application to a Jetson device fleet. The tutorial uses NanoOWL as the example application. NanoOWL is an image analysis optimization project designed to make OWL-ViT models run in real-time on the NVIDIA Jetson Orin platforms. Its real-time capabilities and a flexible “tree detection” pipeline make it ideal for applications like robotics, autonomous vehicles, smart surveillance, industrial automation, and augmented reality experiences where quick object detection and identification are crucial.

Once installed, deploy new AI applications for yourself by following the tutorial, Provisioning NanoOWL into your Jetson board by using Mender to enable real-time, high-performance computer vision at the edge.

In addition, full A/B updates for NVIDIA Jetson are already supported with Mender, both covering L4T (Ubuntu based) and OE4T (Yocto Project based).

Please note: JetPack 6 Development Preview (required for Metropolis on Jetson) does not yet support A/B OTA updates. NVIDIA will support this in the general availability (GA) release.

Mender combined with NVIDIA Jetson addresses the top two use cases. (Stay tuned! More use cases are coming.

In many real-world AI applications – such as smart cameras in physical security, manufacturing, and automotive industries – the AI models or applications must run on edge IoT devices to support rapid response times and efficiency. In collaboration with NVIDIA, the Mender OTA infrastructure now enables full fleet management capabilities with Metropolis Microservices for Jetson. The Mender and Metropolis Microservices for Jetson combination allows AI models, applications, and devices to be easily, quickly and securely updated and managed at scale in production.