# NVIDIA’s BioNeMo Inference Runtime: A New Era in Accelerated Protein Structure Prediction
## The Shift to Proteome-Scale Prediction
Protein structure prediction has entered a new phase. The question is no longer whether machine learning models can accurately fold a single protein — modern deep learning architectures have long since proven that capability. The real challenge today lies in throughput: how efficiently can an entire queue of thousands of independent targets be processed from raw sequence data through parsing, feature generation, GPU-accelerated inference, and final structure output? This is the problem that NVIDIA set out to solve with its latest offering in computational biology.
The scale of modern proteome work is staggering. In one recent large-scale effort, researchers needed to generate structure predictions across nearly 4,800 distinct proteomes, encompassing roughly 31 million candidate protein complexes. Of those, over 1.8 million were released as high-confidence structural predictions — representing one of the largest computational biology pipelines ever executed. Running this kind of work demands an inference infrastructure that is not only accurate but also relentlessly efficient.
## What Is BioNeMo Inference Runtime?
BioNeMo Inference Runtime — commonly referred to as BioIR — is a Python library designed to accelerate supported structure-prediction models on NVIDIA GPUs. Crucially, it does this while preserving the standard PyTorch workflow that researchers and engineers are already familiar with. Models remain ordinary `torch.nn.Module` objects, meaning there is no engine build step, no separate export artifact, and no break in the existing model development pipeline. A checkpoint goes straight into a forward pass without any intermediate conversion.
BioIR targets the specific computational kernels and pipeline stages that general-purpose inference frameworks leave on the table. These include the Pairformer and Evoformer attention stacks, triangle operations used for enforcing geometric consistency in structure representations, pairwise attention mechanisms, diffusion transformer layers, and atom-level modules responsible for coordinate generation. By optimizing these building blocks with GPU-aware custom implementations, BioIR delivers meaningful speedups without requiring users to abandon their preferred frameworks.
The library is available now as an open-source repository with precompiled GPU kernels (CUBINs) distributed via a standard Python wheel. Deployment requires Python 3.12 or newer, a compatible NVIDIA GPU with appropriate drivers, a staged model checkpoint, and per-chain multiple sequence alignments in A3M format. Notably, BioIR does not depend on the CUDA toolkit, CMake, or NVIDIA’s `nvcc` compiler, which dramatically reduces the engineering overhead for setting up a production inference environment.
### How It Is Used
There are two primary modes of interaction with BioIR. The first is the end-to-end processor, which takes an `InputRequest` object and walks it through five sequential stages: parsing input sequences and chain identifiers, tokenizing chains into model-ready representations, generating input features, running accelerated GPU inference, and finally writing PDB or mmCIF structure files along with confidence scores in JSON format. This processor path is demonstrated with Boltz-2 and is designed for users who want a turnkey pipeline.
The second approach is direct PyTorch integration. Developers can instantiate a supported model directly or selectively incorporate individual optimized modules into their own custom code. This is useful for teams that have specific pipeline requirements or wish to mix BioIR-accelerated components with other parts of their workflow.
Each protein chain requires its own A3M-format multiple sequence alignment. Paired or unpaired MSAs are both accepted for inputs containing multiple non-identical protein chains. Templates can be supplied manually, since BioIR does not include HHsearch or HMMsearch functionality for automatic template retrieval. The processor supports ligand structure prediction, though it does not yet handle ligand-affinity prediction tasks.
## Three Layers of Acceleration
BioIR’s optimization strategy operates at three distinct levels, each addressing a different bottleneck in the inference pipeline.
**Layer 1: Kernel Selection**
At the foundation, BioIR performs automatic kernel selection. When a supported operation is invoked, the runtime inspects the model configuration, the target GPU architecture, the data type in use, and the shapes of the involved tensors. Based on these parameters, it selects the most appropriate kernel implementation from three available tiers: BioIR’s own custom CUDA kernels, cuEquivariance kernels optimized for equivariant neural network operations, or a standard PyTorch fallback for operations where a specialized kernel does not apply. This ensures that every layer of the model runs on the fastest available implementation without requiring manual tuning.
**Layer 2: Module Optimization**
Beyond individual kernel selection, BioIR offers a module-level optimization path through a dedicated `optimize()` mechanism. This feature enables CUDA Graph capture for compatible modules. CUDA Graphs record a sequence of GPU operations into a reusable graph that can be launched repeatedly with minimal CPU-side overhead. In the context of structure prediction — where many forward passes share similar computational shapes — this eliminates repeated kernel launch costs and significantly reduces iteration overhead across large work queues.
**Layer 3: Pipeline Scaling with Ray**
The third layer addresses cluster-level throughput. BioIR integrates with Ray to create a replica-scaled inference pipeline. Each visible GPU in a node hosts exactly one complete model replica, and independent input records are distributed across these replicas for parallel processing. The CPU-intensive stages — parsing, feature generation, and output writing — overlap with GPU folding, ensuring that GPUs are kept busy rather than waiting on I/O-bound operations.
It is important to note that Ray replica mode scales independent work items, not individual forward passes. A single protein is folded entirely on one GPU; the system does not split a forward pass across multiple GPUs. However, when hundreds or thousands of proteins need to be processed, distributing them across many GPUs running full model replicas is the most effective strategy. Context-parallel folding, which would split a single large protein across GPUs, is in the roadmap but not yet available in the current release.
The capacity rule is straightforward: the product of compute load per replica and the number of GPUs must not exceed the number of visible GPUs on the node. This keeps resource allocation predictable and avoids oversubscription.
## Benchmark Performance
To characterize the real-world impact of these optimizations, NVIDIA conducted a matched benchmark on a dataset of 1,000 human dimer protein targets. The combined sequence lengths of all targets stayed below 2,800 residues. Both the BioIR-accelerated Boltz-2 implementation and a reference `torch.compile`-optimized open-source Boltz-2 were run on 8xH100 80GB GPUs, using identical targets, staged multiple sequence alignments, and the same inference recipe (three recycling steps, 200 sampling steps, and five diffusion samples per target).
The BioIR implementation completed all 1,000 targets and delivered a throughput of **58,500 successfully folded residues per allocated GPU-hour**. The reference implementation managed only **20,200 residues per GPU-hour** and, critically, ran out of memory on 29 of the 1,000 targets. This translates to a **2.90x improvement** in residue-normalized throughput, along with improved reliability — zero out-of-memory failures versus 29 in the baseline.
It is worth emphasizing that these numbers are folding-stage measurements specific to this dataset and hardware configuration. They exclude MSA generation, CPU-side preprocessing, storage I/O, data transfer between host and device, and any retry or error-recovery overhead. The team behind the work explicitly cautions against extrapolating these figures to all BioIR-supported models or to arbitrary datasets without further benchmarking.
## Energy Efficiency at Scale
Throughput improvements translate directly into energy savings when running proteome-scale campaigns. Extrapolating the benchmark results linearly to 1 million comparable protein targets, BioIR is estimated to require approximately **11 megawatt-hours (MWh)** for folding operations when running on 8-GPU nodes at their thermal design power (TDP) ratings. The reference implementation, by contrast, would consume roughly **35 MWh** — more than three times as much.
If instead of TDP ratings, full-node maximum power draw is used as the baseline, the estimates rise to 21 MWh for BioIR versus 64 MWh for the reference implementation. In either scenario, the energy savings range from 23 to 43 MWh per million targets processed. While these figures are rated-power estimates for IT equipment rather than metered field measurements, and they exclude data center overhead such as power usage effectiveness (PUE) factors, they illustrate a meaningful cost and carbon footprint reduction for organizations running large-scale structural biology pipelines.
## Key Takeaways
– **BioIR accelerates Boltz-2, OpenFold2, and OpenFold3 inference** on NVIDIA GPUs while remaining fully compatible with standard PyTorch workflows. There is no model conversion or engine abstraction layer.
– **A matched 8xH100 benchmark demonstrated a 2.90x throughput gain**, with BioIR processing 58,500 folded residues per GPU-hour compared to 20,200 for the reference implementation — and without any out-of-memory failures.
– **Ray replica mode handles independent worklist scaling**, distributing whole model copies across GPUs to maximize cluster utilization without splitting individual forward passes.
– **Energy consumption at scale drops significantly**, from an estimated 35 MWh to 11 MWh per million targets at 8-GPU TDP equivalents.
– **Production-scale validation already exists**: BioIR powered the AlphaFold Database expansion that generated 31 million candidate protein complexes across 4,777 proteomes.
## Frequently Asked Questions
**Q: What hardware do I need to run BioIR?**
A: You need a Python 3.12+ environment, a compatible NVIDIA GPU with appropriate drivers, and sufficient GPU memory to hold a full model replica. The library ships with precompiled kernels, so no CUDA toolkit or compiler installation is required.
**Q: Can I use BioIR with models other than Boltz-2?**
A: Yes. BioIR supports inference acceleration for Boltz-2, OpenFold2, and OpenFold3. The kernel selection system is designed to adapt to model-specific computational patterns within this family of structure-prediction architectures.
**Q: Does BioIR handle MSA generation automatically?**
A: No. BioIR does not run HHsearch or HMMsearch for template and MSA generation. Users must provide pre-computed A3M-format multiple sequence alignments, one per protein chain, as part of the input request.
**Q: Can BioIR split a single protein’s folding computation across multiple GPUs?**
A: Not currently. Ray replica mode assigns one complete model copy to each GPU and distributes different protein targets across GPUs. Context-parallel folding, which would partition a single large protein’s computation, is planned for future releases.
**Q: How does BioIR compare to simply using torch.compile?**
A: While `torch.compile` provides a general-purpose PyTorch optimization path, BioIR goes further by incorporating model-aware kernel selection, CUDA Graph capture for repeated inference, and pipeline-level scaling via Ray. The benchmark results show roughly 2.9x higher throughput for BioIR versus a `torch.compile` baseline on matched hardware and workloads.
**Q: Is BioIR suitable for production deployment?**
A: Yes. BioIR is already available as an open-source GitHub repository with a distributable wheel package. It has been validated at production scale, including the generation of millions of protein complex structures for a major public structural biology database.
**Q: Does BioIR support ligand-based predictions?**
A: BioIR supports ligand structure prediction. However, ligand-affinity prediction — predicting how strongly a ligand binds to a target protein — is not within its current scope.
## Conclusion
BioNeMo Inference Runtime represents a significant engineering contribution to the field of computational structural biology. By addressing the specific bottlenecks in protein folding inference — from GPU kernel efficiency to pipeline-level parallelism — it enables researchers and institutions to process proteome-scale work queues at a fraction of the time and energy cost of conventional approaches. Its design philosophy of staying within the standard PyTorch ecosystem lowers the adoption barrier considerably: teams can integrate BioIR without retraining models, rewriting inference code, or investing in specialized compilation infrastructure.
The benchmark results are compelling: nearly three times the throughput of a well-optimized baseline implementation, with fewer memory-related failures, and a corresponding reduction in energy consumption at scale. As proteome-level structural biology continues to grow — driven by both academic research and drug discovery programs — tools like BioIR that make large-scale inference practical and energy-efficient will become increasingly essential.
The library is open, well-documented, and ready for deployment today. For teams already using PyTorch-based structure prediction workflows, BioIR offers a clear path to higher throughput with minimal migration effort.
Thank you for reading
