## Understanding GPU Sharing on Kubernetes: The Role of HAMi and Dynamic Resource Allocation
In the evolving landscape of Kubernetes, efficiently sharing GPUs across multiple workloads has become increasingly critical. Projects like HAMi have pioneered sophisticated mechanisms for fractional GPU sharing, but with the advent of Kubernetes Dynamic Resource Allocation (DRA), questions about relevance and synergy naturally arise. This article breaks down these concepts, examines their interplay, and offers practical insights for choosing the right approach.
—
### **How GPU Sharing Worked When the API Couldn’t Help**
Before DRA, sharing GPUs on Kubernetes required workarounds due to limitations in the Device Plugin API. Users had to rely on extended resources like `nvidia.com/gpu`, which treated a GPU as an all-or-nothing resource. HAMi addressed these limitations with a full pipeline including:
– A mutating webhook.
– A custom scheduler extender.
– Annotations to encode allocation details.
– An in-container enforcement mechanism via `libvgpu.so`.
This setup allowed fine-grained resource requests (e.g., specific amounts of GPU memory and compute). However, these advancements were constrained by the underlying API’s inability to natively understand such requests, necessitating opaque string-based annotations and custom scheduling logic.
—
### **What DRA Changes Underneath**
Dynamic Resource Allocation (DRA), available since Kubernetes v1.34 and enabled by default from v1.35, replaces the integer-counting model with a claims-based approach inspired by PersistentVolumeClaims. Key objects include:
– **ResourceSlice**: Describes hardware details per node.
– **DeviceClass**: Defines categories of devices.
– **ResourceClaim & ResourceClaimTemplate**: Specify and instantiate device requests by class, constraints, and capacity.
DRA centralizes allocation decisions, making them visible and manageable through standard Kubernetes APIs. This solves many of the transparency and scalability issues inherent in annotation-based methods.
—
### **Consumable Capacity: Bridging the Gap**
While core DRA provides scheduling visibility, it doesn’t inherently support fractional sharing. This is where **Consumable Capacity** (beta and stable since v1.36) becomes crucial. It allows:
– Multiple independent claims on a single device.
– Capacity requests for specific resources like memory or compute.
For HAMi users, this means:
– `nvidia.com/gpumem: 8000` translates to a memory capacity request.
– `nvidia.com/gpucores: 10` maps to compute capacity.
Under DRA, enforcement still relies on **HAMi-core**, a C library preloaded into containers to intercept CUDA and NVML calls, ensuring compliance at runtime.
—
### **Scheduling Is Still Only Half the Problem**
DRA handles request scheduling but doesn’t enforce runtime constraints. For example:
– A process making unchecked `cudaMalloc()` calls can violate resource guarantees.
– HAMi-core ensures per-container enforcement, maintaining isolation even when neighboring workloads exceed limits.
The ideal architecture combines:
– **DRA** for scheduling and allocation tracking.
– **HAMi-core** for runtime enforcement.
– A compatible DRA driver to integrate the two.
—
### **The DRA Stack HAMi Ships Today**
HAMi’s DRA implementation spans multiple repositories, each addressing specific challenges:
1. **k8s-dra-driver**:
– Publishes GPU capacity via consumable resources.
– Handles allocation on nodes and integrates enforcement.
2. **HAMi-DRA**:
– A mutating admission webhook translating legacy extended resource manifests into DRA-compatible `ResourceClaim` objects.
– Backward-compatible and scheduler-agnostic, supporting platforms like Volcano and KAI.
3. **HAMi**:
– Offers DRA as an installation mode alongside traditional setups.
– Provides observability through Prometheus metrics.
—
### **Key Prerequisites for DRA Mode**
To use DRA, ensure:
1. Kubernetes v1.34+ with `DRAConsumableCapacity` enabled.
2. A CDI-capable runtime (e.g., containerd or CRI-O) and NVIDIA driver 440+.
3. A compatible DRA driver for your hardware.
**Important Note**: DRA and traditional device-plugin modes cannot coexist in the same cluster.
—
### **Comparing the Old and New Approaches**
| Feature | Device Plugin Mode | DRA Mode |
|———————————-|———————————-|————————————-|
| Request Language | Extended resources, opaque | ResourceClaim capacity requests |
| Scheduling Decision | HAMi scheduler extender | Native kube-scheduler |
| Enforcement | HAMi-core (libvgpu.so) | HAMi-core (unchanged) |
| Kubernetes Version Compatibility | All supported versions | v1.34+; stable from v1.36 |
| Runtime Prerequisites | Standard container runtime | CDI-capable runtime, NVIDIA driver |
—
### **How to Choose: Recommendations for Mid-2026**
– **Stick with traditional mode if**:
– You’re on managed Kubernetes without feature-gate access.
– You’re using mixed accelerators not yet supported by DRA drivers.
– Long-term stability and extensive vendor coverage are priorities.
– **Adopt DRA mode if**:
– You’re on Kubernetes v1.36+ or manage your own control plane.
– You want native scheduler integration and improved observability.
– You’re willing to test pre-stable features in exchange for future-proofing.
—
### **Conclusion**
GPU sharing on Kubernetes has come a long way, transitioning from opaque workarounds to standardized mechanisms like Dynamic Resource Allocation. While DRA addresses much of what HAMi originally solved, it complements rather than replaces the need for runtime enforcement. By combining DRA for scheduling and HAMi-core for enforcement, users can achieve both efficiency and reliability in multi-tenant GPU environments. For organizations navigating this transition, understanding these tools and their synergies is key to unlocking scalable, high-performance Kubernetes.
