All of you know about Kubernetes, the open-source container-orchestration system for automating application deployment, scaling, and management. Over time, it is becoming the de-facto standard to automate the deployment, scaling, and operations of container-based workloads in the cloud. The Cloud-Native Computing Foundation (CNCF) is responsible for the certification of the Kubernetes distributions to provide a compatible environment across different cloud-blade and on-premises installations.
One of the most interesting Kubernetes distributions is k3s from Rancher, a lightweight certified Kubernetes distribution specifically built for IoT and Edge computing. Still retaining full Kubernetes compatibility and an impressive feature set, the single binary which provides k3s is less than 40MB and is available for both ARM64 and ARMv7.
k3s provides the ability to IoT and Edge computing engineers to deploy and manage container-based applications into their fleet of devices describing their application workload using the Kubernetes manifesto files, a collection of YAML files describing the desired state of the deployment. It's a future-proof, feature-richer, and more expressive alternative to manage containerized applications in the field using docker-compose, the ubiquitous tool used by developers approaching the Docker world.
By default, it uses an sqlite3 database as the lightweight backend storage. It optionally supports other backend storage, namely, etcd, MySQL and PostgreSQL, which are less suitable for embedded IoT and Edge devices, so they are out of scope for this article.
Curiosity about the naming: we abbreviate Kubernetes as k8s (k and s with eight letters in the middle, making it a ten characters word). When developing k3s, Rancher had the goal of creating a Kubernetes distribution that could run with half the resources needed to run the stock Kubernetes. As half of ten is five, they named k3s (k and s with three letters in the middle, making it a five characters word). Nobody knows what the three middle letters are, though. :)
Getting started with k3s
Installing k3s on a device, as small as a Raspberry PI or as big as a 32 CPU virtual server running in the cloud, is a matter of executing the following command:
$ curl -sfL https://get.k3s.io | sh -
[INFO] Finding release for channel stable
[INFO] Using v1.18.2+k3s1 as release
[INFO] Downloading hash https://github.com/rancher/k3s/releases/download/v1.18.2+k3s1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/rancher/k3s/releases/download/v1.18.2+k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s
In less than a minute, our single-node Kubernetes cluster is ready to run our containers. We can use the stock Kubernetes kubectl CLI get the list of pods, the smallest deployable units of computing you can start, running:
$ kubectl get pods
No resources found in default namespace.
Install and upgrade applications in k3s using Mender
Mender is our robust and secure over-the-air software updater for IoT devices. We support the installation and upgrade of containerized workloads running on Kubernetes, including k3s, through the Kubernetes Update module.
The easiest way to try the Mender OTA platform is by signing up for the managed service. The Mender Starter plan offers a three-month evaluation period free of charge.
Package k8s/k3s configuration as Mender artifacts
First of all, let's define a sample application workload which runs the stock nginx Docker image by creating the file nginx-deployment.yaml as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
We can now use the k8s-artifact-gen utility to generate a Mender Artifact, which we'll be able to deploy using the Mender OTA platform, from the Kubernetes manifest file defined above:
$ wget https://raw.githubusercontent.com/mendersoftware/mender-update-modules/master/k8s/module-artifact-gen/k8s-artifact-gen
$ chmod 775 k8s-artifact-gen
$ ./k8s-artifact-gen \
-n nginx-sample-app \
-t device_type \
-o nginx-sample-app.mender \
nginx-deployment.yaml
Artifact nginx-sample-app.mender generated successfully:
Mender artifact:
Name: nginx-sample-app
Format: mender
Version: 3
Signature: no signature
Compatible devices: '[device_type]'
Provides group:
Depends on one of artifact(s): []
Depends on one of group(s): []
State scripts:
Updates:
0:
Type: k8s
Provides: Nothing
Depends: Nothing
Metadata: Nothing
Files:
name: nginx-deployment.yaml
size: 316
modified: 2020-05-27 06:17:57 +0200 CEST
checksum: 529d78bd4aac18e2343426fa6267960a74f3c5a450ab04f32c4acd42efb9bdfe
Please note: you have to replace the string device_type with the device type you set for your own device, for example beaglebone or raspberrypi3. For more information about setting the device type, please refer to the official Mender documentation.
The file nginx-sample-app.mender is the Mender Artifact that embeds the Kubernetes manifest file(s) that you can deploy into your k3s devices using the Mender OTA platform.
You can upload the artifact using the Mender user interface or using the mender-cli, running:
$ mender-cli login --server https://hosted.mender.io --username your@email.com
Password:**********
login successful
$ mender-cli artifacts upload nginx-sample-app.mender --server https://hosted.mender.io
Processing uploaded file. This may take around one minute.
upload successful
Install the Mender Client and the k8s Update Module on the device
To deploy and upgrade Kubernetes manifests on a device running Kubernetes, you need to install the Mender client. You can follow the instructions in our documentation page to Install the Mender client.
We are going to use the k8s Update module to deploy our application workload, described in the Mender jargon as Application updates. As full system updates are out of scope for this article, we can easily install the Mender client using a Debian package as described in the Install Mender using the Debian package documentation page.
Once the Mender client is installed and authenticated the installed the Mender client and authenticated it to connect to our Mender server, either on-premises or our hosted Mender service,. we are ready to install the k8s Update module on the device:
$ mkdir -p /usr/share/mender/modules/v3
$ wget -P /usr/share/mender/modules/v3 https://raw.githubusercontent.com/mendersoftware/mender-update-modules/master/k8s/module/k8s
$ chmod +x /usr/share/mender/modules/v3/k8s
Deploy a Kubernetes update using Mender
At this point, we can create a deployment from the Mender graphical user interface for our device, and the Mender client will apply the Kubernetes manifest file using the k8s Update module.
Internally, the k8s Update module uses the kubectl CLI to apply the manifesto files bundled in the Mender artifact.
We can finally verify the deployment is in place on our device:
$ kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 1/1 1 1 74s
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-deployment-6b474476c4-kzq66 1/1 Running 0 83s
Conclusion
Containers are gaining a lot of traction in the embedded world. The most common and well-known path, to deploy containerized applications in IoT and Edge devices is using either Docker directly or docker-compose. Still, Kubernetes represents a valuable alternative for use cases where, despite the limited resources, a full container-orchestrator can provide developers a stateful environment to run platform-agnostic workloads.
