# NVIDIA CUDA Rust: A New Era for GPU Kernel Development in Rust
## Introduction
The landscape of AI systems programming is shifting rapidly, and NVIDIA has taken a significant step in bridging the gap between safe systems languages and GPU programming. The announcement of CUDA Rust introduces two open-source projects designed to make Rust a first-class language for writing GPU kernels, opening up new possibilities for developers who want memory safety guarantees without sacrificing the performance of GPU-accelerated computation.
## Why Rust for GPU Kernels?
For years, Rust has been gaining ground in the systems layer of artificial intelligence — from inference engines and runtime environments to device drivers. NVIDIA’s own developments reflect this trend, with components like the Nova Linux driver, the Dynamo orchestration layer, and NVTX profiling bindings all embracing Rust. However, the GPU kernel remained a notable exception, with developers still relying on CUDA C++ or Python-based solutions for writing the actual parallel computation code that runs on the GPU.
This gap matters because GPU kernels are where the most performance-critical and bug-prone code lives. Memory safety issues in kernels can lead to silent data corruption, security vulnerabilities, and extremely difficult-to-debug failures. Rust’s ownership model and borrow checker offer a compelling solution — but only if they can be brought to bear on the unique challenges of GPU programming.
## The Two Programming Models
CUDA has long supported two distinct programming models, and the Rust push mirrors both of them:
### SIMT (Single Instruction, Multiple Threads)
This is the traditional CUDA model familiar to C++ and Python developers using frameworks like numba-cuda. In SIMT, you describe what a single thread does, and the CUDA runtime launches thousands of those threads in parallel. Each thread operates on its own piece of data, and developers have explicit control over thread indexing and memory access patterns.
### Tile Model
The newer Tile model takes a higher-level approach. Instead of describing individual threads, you describe what a tile — a chunk of data — does, and a compiler intermediate representation handles the details of mapping work to threads and managing memory layout. NVIDIA recommends starting with the Tile model for most use cases, reserving SIMT for situations where explicit thread and memory control is necessary.
A key advantage of supporting both models is that planned inter-language interop means choosing Rust will not lock developers out of continuing to use C++ or Python where those are more appropriate.
## The SIMT Track: cuda-oxide
cuda-oxide is a custom `rustc` code generation backend that enables writing GPU kernels in pure Rust using the SIMT programming model. Here is how it works:
– Kernel functions marked with the `#[kernel]` attribute are routed through Rust’s Middle Intermediate Representation (MIR).
– From MIR, they pass through the community-developed Pliron IR framework, which includes NVIDIA-specific GPU dialects.
– The standard LLVM backend then handles the final lowering to PTX (Parallel Thread Execution), CUDA’s native GPU assembly language.
– The resulting PTX code is embedded into a device bundle and launched through a checked contract that validates the configuration before execution.
### Requirements
cuda-oxide is currently in early alpha and has specific requirements:
– Linux operating system
– A GPU with compute capability 8.0 or later
– CUDA 12.x or newer
– clang with libclang installed
– A pinned nightly Rust toolchain (`nightly-2026-04-03`)
Setting up a project is straightforward: `cargo oxide doctor` checks your environment, and `cargo oxide new` scaffolds a vector addition program with both host and device code in a single file.
### Safety in the Kernel Signature
The safety argument becomes tangible in how kernel inputs and outputs are typed. Regular input parameters are passed as ordinary shared slices, while output parameters use a `DisjointSlice
Bounds checking is built in: `c.get_mut(idx)` returns an `Option`, so out-of-bounds access becomes a handled branch rather than undefined behavior. The `#[launch_contract]` attribute declares the block shape, and the generated launch method validates the configuration before any GPU work begins.
## The Tile Track: cutile-rs
cutile-rs takes a fundamentally different and more abstracted approach. Rather than generating PTX directly, it captures the kernel’s abstract syntax tree (AST) and embeds it in the host binary, then JIT-compiles it through the CUDA Tile IR compiler when the kernel is first launched.
### How It Works
In cutile-rs, each tile block runs the kernel body once as a single logical thread operating over one sub-tensor. The compiler decides how many real GPU threads back each tile, abstracting away the low-level thread management from the developer.
The setup is remarkably simple: `cargo new` creates a fresh project, and `cargo add cutile` adds the dependency. There is no need for a nightly toolchain or custom LLVM — cutile-rs works with stable Rust 1.89 or newer.
### Ownership and Partitioning
The host-side `.partition([128])` call performs three critical operations simultaneously. It gives each tile exclusive ownership of its 128-element chunk, fixes the grid dimensions (for example, 1,024 elements divided by 128 per tile yields 8 tiles), and supplies the constant tile width `B` to the kernel. Input tensors can use `-1` as a dynamic dimension that gets resolved at launch time.
The generated launcher takes ownership of all tensors and returns them once the GPU finishes. Nothing executes until `.sync_on(&stream)` is called; everything before that point is a lazy description of work recorded in a single chain. This declarative approach makes it easier to reason about when GPU operations occur and what data they depend on.
### Requirements
cutile-rs has lighter requirements than cuda-oxide:
– Compute capability 8.0 or later
– CUDA 13.3
– Stable Rust 1.89 or newer
– Linux
– No nightly toolchain required
– No custom LLVM dependencies
## Safety Guarantees: Preventing Aliasing Bugs at Compile Time
One of the most compelling aspects of both CUDA Rust tracks is that Rust’s ownership rules are used to reject aliasing bugs at compile time rather than discovering them at runtime or in production. The two tracks enforce safety differently but with the same goal: making certain classes of GPU programming errors impossible to write.
In the SIMT track, the `DisjointSlice` type and launch contracts ensure that output buffers are exclusively accessible and that launch configurations match what the kernel expects. In the Tile track, partitioned tensor ownership follows the data across launch boundaries, making it impossible to accidentally use a tensor after it has been consumed by a kernel launch.
Both approaches catch the exact error class of aliasing — where one buffer is read and written simultaneously by different parts of a kernel — before any GPU code ever runs.
## Deployment Readiness
The two projects are at different stages of maturity:
– **cutile-rs** is published on crates.io and is already being used in production-like scenarios. It powers parts of Hugging Face’s Grout inference engine and is integrated into mistral.rs, a high-performance Rust inference engine for large language models.
– **cuda-oxide** is in early alpha and is not yet confirmed for production use.
Both projects are broadly in alpha phase, meaning they are functional and useful but may undergo significant API changes as they mature.
## Frequently Asked Questions (FAQ)
### Q1: What is CUDA Rust?
A: CUDA Rust refers to the effort to make Rust a first-class language for writing GPU kernels that run on NVIDIA GPUs. It encompasses two open-source projects — cuda-oxide for the SIMT programming model and cutile-rs for the Tile programming model — both designed to compile Rust code directly to GPU-executable code.
### Q2: Can I use CUDA Rust with stable Rust?
A: Yes, but it depends on the track. cutile-rs works with stable Rust 1.89 and newer, making it accessible for production experimentation. cuda-oxide currently requires a pinned nightly toolchain and is in early alpha.
### Q3: What GPU hardware is supported?
A: Both tracks require NVIDIA GPUs with compute capability 8.0 or later. This includes Ampere architecture GPUs (A100, A30, A40) and newer architectures like Hopper (H100, H200) and Blackwell.
### Q4: What is the difference between SIMT and Tile programming models?
A: SIMT is the traditional CUDA model where you describe what a single thread does and the runtime launches thousands of threads. Tile is a higher-level model where you describe what a chunk of data (a tile) does, and the compiler handles the mapping to threads and memory layout. Tile is recommended for most use cases, while SIMT is preferred when fine-grained thread and memory control is needed.
### Q5: Does CUDA Rust replace CUDA C++?
A: Not at this stage. CUDA Rust is an alternative that offers memory safety guarantees. Planned inter-language interop means Rust code can coexist with C++ and Python code, so developers can choose the right tool for each part of their workload.
### Q6: Can I share memory in CUDA Rust kernels?
A: Shared memory is available, but in cuda-oxide (the SIMT track) it currently requires unsafe code. cutile-rs (the Tile track) manages shared memory through the compiler, reducing the burden on the developer.
### Q7: What kind of safety guarantees does CUDA Rust provide?
A: Both tracks use Rust’s ownership system to prevent aliasing bugs at compile time. The SIMT track uses `DisjointSlice` types and launch contracts to ensure exclusive access to output buffers. The Tile track uses partitioned tensor ownership to prevent use-after-move and data races across kernel launches.
### Q8: Where can I find these projects?
A: Both cuda-oxide and cutile-rs are open-source projects. cutile-rs is published on crates.io and can be added to any Rust project with `cargo add cutile`. cuda-oxide is available through the NVIDIA developer ecosystem and is in early development.
## Conclusion
NVIDIA’s CUDA Rust initiative represents a meaningful step toward bringing memory safety to GPU programming. By offering two tracks that match CUDA’s existing programming models, the project gives developers a clear path to adopt Rust for their most performance-critical code without sacrificing the ability to use the full power of NVIDIA GPUs. The fact that cutile-rs is already deployed in real inference engines like Grout and mistral.rs demonstrates that the technology is not merely theoretical — it is being used to accelerate actual AI workloads today.
As both projects mature out of alpha and inter-language interop becomes more robust, we can expect to see Rust gradually replace CUDA C++ as the preferred language for writing new GPU kernels, particularly in safety-critical and security-sensitive applications. The combination of Rust’s compile-time guarantees with the raw computational power of NVIDIA GPUs could set a new standard for how high-performance computing code is written and maintained.
Thank you for reading
