Insights from the Linux Plumbers Conference 2023
The global Linux market is forecasted to reach $15.64 trillion by 2027, and commands usage among nearly half, 47%, of professional developers. Powering the world’s fastest computers, Linux is a core pillar in embedded technology and the future development of the IoT market.
Billed as “the premier event for developers working at all levels of the plumbing layer and beyond,” the Linux Plumbers Conference 2023 delved into the intricacies of Linux in all its forms. Exploring the latest developments in the Yocto Project and the challenges of maintaining and managing device fleets over years and decades, the technical and community-driven tapestry of the conference shed light on key discussions and takeaways that could define the landscape of embedded Linux in the years to come.
1. A common path forward: Ensuring long-term building and maintenance is required.
With the Linux Plumbers Conference, “Plumbers” or “LPC” for short, a self-declared “working conference,” there was an exceptionally strong culture of discussion and knowledge sharing. As the old saying goes, “The hallway track is the best track!” and individuals involved at various stages of putting Linux into embedded platforms shared their insights, learnings, and challenges.
In doing so, a clear takeaway is that the community, being composed of individuals with very diverse backgrounds and goals, has not found a common path forward. While the technological purists proclaim a “stay on master” approach to always follow the latest and greatest, the conservative device manufacturers often lean towards a “ship it and hope for the best '' mindset. This dichotomy manifests itself in many forms.
For example, the Yocto Project offers a four-year long-term stable (LTS). This four-year LTSt is considered way too short by many, and others frown upon the maintenance effort already required. Building a Linux distribution is one element; getting this distribution deployed to devices is a whole different story. As such, Tim Bird's “Embedded BoF” session identified over-the-air (OTA) updates as one of the top current challenges in the Linux ecosystem. There needs to be a common framework for fundamental requirements in leveraging Linux in production environments.
2. Production reality: Device fleet and management at scale remains a challenge.
There are numerous initiatives around making embedded Linux fit for high-profile use cases, including applications in the automotive, avionics, industrial automation, and medical technology sectors. Each industry brings with it its own set of challenges.
One big topic is certifications, such as Software Considerations in Airborne Systems and Equipment Certification (DO-178C) and functional safety compliance (ISO 26262). For example, the ELISA project is currently working on functional safety and ISO 26262. The focus on certifications goes along with a growing interest in heterogeneous or asymmetric multiprocessing (AMP) systems. These systems incorporate not only an MPU-style core CPU (such as a Cortex-A class ARM core) but also a number of separate, sometimes specialized co-processors. The range includes designs for hardened, safety-critical applications (such as the Cortex-R class that can be certified for safety regulations) over more classic MCUs (like RISC-V RV32E or ARM Cortex-M-based ones for cost-efficient logic to highly customized CPU cores in FPGA-based devices that enable high-performance data processing). What they all have in common is that these co-processors usually do not run Linux but rather an RTOS, such as Zephyr or bare-metal code.
Despite the focus on certifications, compliance, and safe operations, a coherent concept of managing the resulting device fleets, especially in an objective method, is lacking. The new challenge with these systems is how to manage software maintenance strategies over prolonged periods of time. Achieving long-term maintenance requires both the continuous building and testing of the MCU firmware blobs. These need to be aligned with approaches for Linux, using either filesystem-level mechanisms or containers; however, all of the current methods of fleet management force the user into a specific mindset. As devices and fleets scale, users require a flexible model in order to be adaptable to nearly every possible software deployment scenario. In order to maintain device fleets over a long period of time, users need to adopt an understanding of how to manage their devices as part of the product as a whole, delivering continuous value to the end customer.
3. The Y2038 problem: Preparations are not being taken throughout the ecosystem.
This may come as a surprise to those not deeply familiar with software, but there is a well-known breaking point for many pieces of code at 03:14:07 UTC on January 19, 2038. This is the so-called Y2038-Problem, named after the year in which it will happen. In short, this is the point in time when the so-called UNIX timestamp will overflow. Based on a 32-bit number counting the seconds since 00:00:00 UTC on January 1, 1970, the time stamp will hit the limit of its value range at that point.
At this moment, all time-based arithmetics happening on a Linux system will fail in various ways. For anyone in the business of embedded systems, there are “only'' 15 years to go, which is not a lot of time. More and more of the systems being built today will live to operate by the Y2038 date. While the core Linux maintainers already have their mitigation strategies in place, this does not hold true for major parts of the software infrastructure, such as libraries and applications. Current designers or device manufacturers should plan for this event accordingly. This does not mean that all code needs to be revisited now. But, there should be awareness of the fact that Linux users will need to deploy updated software components once all the ecosystems and libraries are ready. And, importantly, there must be a robust method to deploy updates to devices at scale.
The future of the embedded Linux ecosystem
The Linux ecosystem and embedded Linux world are stronger than ever. It powers billions of devices with amazing manifestations – moving the majority of worldwide data traffic, allowing helicopters to fly on planet Mars, and operating massive MRT scanners. Linux is a pillar upon which large parts of today's everyday technology is built.
As opposed to the sometimes hyped projects in the cloud world, most embedded systems, embedded Linux development, and maintenance happen in a very small and often resource-constrained group. On one hand, this fact creates a group of people who are strongly connected with each other and share a strong vision of how to move forward. On the other side, however, it also results in a massive imbalance between A) those individuals putting effort, knowledge, and energy into creating a sustainable and robust ecosystem and B) the huge number of devices and companies that just consume these goods.
And the challenges have also increased. Maintaining and managing the plethora of devices out in the field is still an unsolved problem for many. To scale efficiently and securely requires both a strategy to ensure the software pieces in themselves are properly understood and maintained and a plan to manage the logistics and related aspects of deploying changes to devices or fleets of devices already in use. Without these two components, realizing Linux devices in production and at scale will remain stunted by challenges.
As such, Northern.tech, the team behind Mender, believes that open source is one of the foundations of secure devices. And this is why we are active supporters of the Yocto Project ecosystems. There are many ways that any company or organization can support the various projects here: sign for membership, contribute code to the technologies used, or hire people with a passion and give them the freedom to contribute patches to the project. However one chooses to support these projects, it is clear Linux and open source are here to stay and will be foundational to the future of the IoT world.
