Bringing container applications to resource-constrained edge hardware

The challenge: Kubernetes is too complex for small edge systems

Many industrial companies face two fundamental challenges when operating their OT solutions:

• Applications are moving closer to machines and plants, forming an industrial edge.
• There is a growing need to use container applications at the industrial edge as well.

Container applications simplify deployment on heterogeneous hardware and make it easier to update to new versions. In addition, many production sites are not permanently connected to a central cloud or data centre. Applications must therefore keep running locally even when the network is unstable. This leaves companies with a balancing act between a modern application platform and low resource consumption.

In IT, Kubernetes has become the established standard for managing container applications. It offers a large ecosystem of tools, standardised interfaces and ready-made applications. However, the operational complexity of Kubernetes is out of proportion for small devices. This is because the software was originally developed for clusters with multiple compute nodes. Its components also include a so-called control plane as the central management layer, networking functions, storage management and numerous mechanisms for distributed systems.

In short, a Kubernetes environment is normally extensive and resource-intensive. Even the lightweight Kubernetes distributions K3s and MicroK8s have relatively high requirements. Many industrial edge systems, on the other hand, have no need at all for advanced functions such as multi-node clustering or high availability.

Another factor is know-how. Mid-sized companies in particular have extensive OT expertise, but their IT resources are scarce. Even IT departments often lack experience with containers and Kubernetes. What is needed, therefore, is a distribution that works on small systems, provides all the basic functions and can be operated without staff having to become Kubernetes experts.

The solution: Lightweight Kubernetes for single-node operation

Portainer, provider of a platform for container management from the cloud to the industrial edge, is therefore taking an OT-specific approach with KubeSolo, aimed at mid-sized industrial companies. Technically, the system is based on K3s. Portainer has developed this lightweight distribution further into a hardened single-node system for resource-constrained devices.

The approach also reflects the experience that changes to interfaces can quickly bring existing infrastructures to a standstill. When Docker deprecated an older API version, many container environments stopped working. At the time, Portainer released an update within one to two days. With KubeSolo, the company relies on the stable Kubernetes API and is actively developing the distribution further, so that users can always deploy the latest version.

Fewer components reduce resource requirements

For KubeSolo, the main step was to remove or simplify functions that are not required for operation on a single device. These include multi-node clustering, high availability across multiple nodes and the etcd data store. The networking and ingress functions are also tailored to use on a single host, reducing memory requirements and CPU load. At the same time, the core Kubernetes interfaces are retained.

This enables companies to use existing container applications in production without having to run a full cluster at every site. In combination with the Portainer platform, industrial edge systems can also be managed centrally. The result is an architecture that takes the limited resources of industrial devices into account. The memory footprint is under 200 MB of RAM. KubeSolo is well suited to devices with 512 MB of memory and, according to Portainer, has even been successfully tested on devices with 128 and 256 MB. The distribution thus closes the gap between a simple container host running Docker or Podman and a full cluster.

Kubernetes also runs on older industrial hardware

A key feature of KubeSolo is its focus on existing industrial systems. It works on a wide range of hardware and architectures. By conventional IT standards, many industrial systems would be seen as outdated or highly constrained. These include single-core systems, older ARM architectures and devices with very little memory. Yet it is precisely these systems that form an important layer in the Industrial IoT between the sensors on the machines and the higher-level IT systems. For this purpose, KubeSolo offers the following features:

• It supports the common processor architectures x86/x64, ARM and ARM64.
• Modern Linux distributions such as Ubuntu, Arch Linux, Armbian and Siemens Industrial OS can be used as the operating system.
• It has a networking model tailored to single-node operation. All necessary functions of the “Container Network Interface (CNI)” are included. This makes network configuration considerably simpler.
• Tested platforms include NVIDIA Jetson for edge AI, Siemens SIMATIC IOT2050, Raspberry Pi systems, ZimaBoard and Intel Atom platforms. These are joined by Linux-capable systems from Beckhoff, Phoenix Contact and Advantech.
• It can be administered via the standard API, for example with kubectl, OpenLens, Headlamp, K9s or development tools such as VS Code.

From an application’s perspective, KubeSolo behaves like Kubernetes. The API endpoint corresponds to the respective version, and all resource definitions can be reused. Persistent data can also be managed via a local storage provisioner, an important criterion for industry. Interfaces are also available for external storage solutions. In addition, CPU, memory and storage limits can be set for each container. This allows the device’s scarce resources to be allocated in a targeted way.

Another feature is local autonomy. Each device has its own control plane, i.e. its own management layer. It therefore does not have to be part of a cluster as a worker node and can operate autonomously. Moreover, local Kubernetes control is retained even if the connection to headquarters fails. This is particularly relevant for sites that offer only low bandwidth, have high latency or experience frequent connection interruptions. As soon as the connection is restored, telemetry, device status and updates are exchanged with the higher-level layer.

Open source and central management complement each other

KubeSolo is an open-source distribution available in a free and a commercial edition.

• The free edition offers the full feature set of KubeSolo and can be used independently of Portainer. This makes it suitable for individual systems at the industrial edge. In this case, support is provided by the community.
• The commercial edition combines KubeSolo with the Portainer platform. It is aimed primarily at companies that want to operate larger fleets of distributed edge and Industrial IoT systems centrally and use commercial support. The system runs as a standalone single-node runtime environment on each device. Portainer handles the central management of all instances.

The result: Edge systems become a Kubernetes platform

From a business perspective, KubeSolo above all makes better use of systems that are already installed and simplifies getting started with Kubernetes. With KubeSolo, existing hardware does not have to be replaced by more powerful systems. Applications run on devices that would be too weak for other Kubernetes distributions. Companies also gain access to the Docker and Kubernetes ecosystems. Instead of developing functions such as data aggregation, visualisation or protocol conversion themselves, they can deploy ready-made solutions as containers. A proof of concept then often consists of nothing more than configuring a container.

This turns an individually maintained device into a standardised Kubernetes endpoint that remains autonomous when connection problems occur. The system is therefore well suited to production sites, plants or other devices with unstable or deliberately restricted connectivity.

At the same time, Portainer forms a central layer for management and governance. The platform monitors, configures and controls many autonomous KubeSolo instances from a single higher-level interface. Kubernetes compatibility also expands the range of available applications.

In addition, simple installation and updating keep operating effort low. KubeSolo is provided as a single executable file (binary) and is easily updated to a new version by replacing that file. Installation scripts make it possible to set up new environments quickly. This simplicity is crucial, especially in larger fleets: every additional configuration step for a single edge device adds up to a considerable total effort across hundreds or thousands of instances.

KubeSolo in practice

Initial deployment scenarios show how KubeSolo can be used for data acquisition and analysis in industrial environments. In an offshore project in the oil and gas industry, the technology is currently being trialled as a possible replacement for existing Docker endpoints. These use fieldbuses to acquire measurement data from current transformers in switchgear and make it available for monitoring energy consumption.

In a further step, the platform could also replace the data aggregation system currently deployed at each site, process measured values locally and pass relevant data on to higher-level IT systems.

At another company, KubeSolo is already in productive use, among other things to reduce licence costs for existing VM infrastructures. A distributed architecture in which each site operates autonomously is also being planned there. Updates are to be installed locally even when there is temporarily no connection to the central infrastructure.

Further trials are under way in manufacturing, where an industrial company is deploying the platform with test licences directly on bare-metal systems. In addition, several major manufacturers of automation and process control systems are evaluating the technology internally as a possible basis for their own products.

Portainer is continuously developing KubeSolo further. Plans include provisioning new devices with KubeSolo via Portainer straight out of the box and integrating graphics processing units (GPUs) for AI applications at the edge during installation.