**HAMi: Cloud-Native GPU Virtualization for Kubernetes Moves to CNCF Incubation**
The CNCF Technical Oversight Committee (TOC) has voted to accept HAMi as a CNCF incubating project, marking a significant milestone for the open-source GPU virtualization platform. This progression from Sandbox to Incubation reflects strong community adoption and technical maturity, positioning HAMi as a key enabler for heterogeneous compute scheduling in Kubernetes environments.
—
### About HAMi
Modern AI infrastructure teams face a recurring challenge: expensive GPUs often sit fragmented and underutilized because entire devices are allocated to workloads that need only a fraction of their capacity. At the same time, teams compete for scarce accelerators, and different hardware vendors expose distinct operational models. Effective scheduling of these heterogeneous accelerators requires device-level context that extends beyond the needs of general-purpose compute.
HAMi addresses these challenges by providing an open-source, cloud-native GPU virtualization middleware for Kubernetes. It allows platform teams to:
– Slice physical GPUs (and other accelerators such as NPUs, DCUs, or MLUs) by memory, core, or device count.
– Enforce hard runtime isolation between workloads sharing a device.
– Apply binpack, spread, and topology-aware scheduling policies.
– Achieve all of this without modifying application code or existing Kubernetes resource manifests.
Unlike device-plugin-based approaches tied to single vendors, HAMi offers a multi-vendor design with a consistent interface, making it a vendor-neutral solution for diverse accelerator landscapes.
—
### Key Milestones and Ecosystem Growth
Since joining the CNCF Sandbox on August 21, 2024, HAMi has demonstrated substantial progress across adoption, contribution, and real-world deployment.
The project boasts over 550 contributing organizations and has been featured in five independent CNCF case studies. These highlight production use in sectors such as education, cloud platforms, and enterprise technology. Notable deployments include:
– DaoCloud’s rollout of HAMi across more than 10,000 GPUs in over 10 data centers in mainland China and Hong Kong.
– China Merchants Bank’s use of HAMi to manage diverse accelerator resources at scale.
With approximately 3,500 GitHub stars and over 550 forks, HAMi has seen explosive community engagement, including 2,687 contributors and a 43% year-over-year increase in contributions. The project has shipped 16 releases, with the current stable version at v2.9.0.
HAMi continues to integrate deeply within the CNCF ecosystem, collaborating with projects such as:
– Volcano for batch-oriented AI scheduling.
– Koordinator for GPU-sharing workflows.
– Kueue (and others) to help build a more complete cloud-native AI infrastructure stack.
—
### HAMi’s Main Components
HAMi is composed of several core components:
– **Mutating Webhook**: Intercepts pod submissions in the Kubernetes API server and adjusts scheduler fields and resource requests for virtualized devices.
– **Scheduler Extender**: Handles filtering, scoring, and binding of pods to nodes and devices using advanced placement policies.
– **Device Plugins**: Vendor-specific plugins that register accelerators with Kubernetes and allocate fractional device resources.
– **HAMi-Core**: The in-container virtualization layer that enforces hard runtime limits on GPU memory and compute, intercepting the native CUDA driver for NVIDIA devices.
– **HAMi-WebUI**: A visual interface for cluster and device management, offering visibility into allocation and utilization.
– **Observability Layer**: A Prometheus-compatible metrics endpoint and Grafana dashboards for cluster-wide monitoring.
—
### HAMi’s Roadmap
The HAMi team is focused on advancing scheduling capabilities, including:
– Gang-scheduling
– Preemption
– Autoscaling
The project is also working toward solutions for monitoring Dynamic Resource Allocation (DRA) consumption and expanding device support to include AMD Mi Series and PPU accelerators. Further collaboration with other CNCF scheduling projects is planned to strengthen the cloud-native AI infrastructure landscape.
—
### FAQ
**What problem does HAMi solve?**
HAMi solves the challenge of underutilized and inefficiently scheduled GPUs and other accelerators in Kubernetes environments. It enables fine-grained sharing of hardware devices, multi-vendor support, and simplified management through virtualization and advanced scheduling policies.
**How is HAMi different from Kubernetes Device Plugins?**
Unlike device plugins, which are typically vendor-specific and attach directly to kubelet, HAMi provides a vendor-neutral virtualization layer. It allows fractional allocation, runtime isolation, and flexible scheduling policies without requiring changes to application code.
**What accelerators does HAMi support?**
HAMi supports a range of accelerators, including NVIDIA GPUs, as well as NPUs, DCUs, and MLUs. The project is also working to expand support to include AMD Mi Series and PPU accelerators.
**How does HAMi integrate with Kubernetes scheduling?**
HAMi operates as a scheduler extender and mutating webhook, seamlessly integrating with both the default Kubernetes scheduler and batch-oriented schedulers like Volcano. It works alongside existing Kubernetes configurations without requiring disruptive changes.
**What are the benefits of HAMi for AI workloads?**
HAMi improves GPU utilization, reduces contention for accelerator resources, and provides consistent operations across heterogeneous hardware. These benefits are especially valuable for AI and machine learning workloads running at scale in Kubernetes.
—
### Conclusion
HAMi’s transition to CNCF Incubation underscores its technical strength, growing ecosystem, and real-world impact in AI infrastructure management. By delivering a vendor-neutral virtualization layer for heterogeneous accelerators, HAMi helps organizations get more from their hardware, streamline scheduling, and avoid vendor lock-in. As it moves toward graduation, HAMi is well positioned to become a foundational component of the cloud-native AI stack, supported by a vibrant and expanding community. For those looking to optimize GPU usage in Kubernetes, HAMi offers a promising path forward.
