**LingBot-VLA 2.0: A Generalist Vision-Language-Action Foundation Model for Robots**
Ant Group’s Robo-Agent team has open-sourced **LingBot-VLA 2.0**, a large-scale Vision-Language-Action (VLA) foundation model designed to serve as a generalist robot policy. Released alongside a technical report and an Apache-2.0 codebase, LingBot-VLA 2.0 directly targets real-world deployment challenges that have historically limited lab-only VLA models. The update delivers stronger generalization, a broader action space, and predictive dynamics—bridging the gap between simulation performance and real-robot success.
### What is LingBot-VLA 2.0?
At its core, LingBot-VLA 2.0 is a vision-language-action model that maps camera images and language instructions to robot actions. The public checkpoint, `lingbot-vla-v2-6b`, is built on the Qwen3-VL-4B-Instruct vision-language model and uses a “native depth” design. Two teacher models—LingBot-Depth (geometric depth) and DINO-Video (temporal semantic priors)—supervise training via distillation. A single forward pass on an NVIDIA GeForce RTX 4090D takes about 130 ms (with 10 denoising steps), making real-time control feasible. The action expert employs a Mixture-of-Experts (MoE) architecture to scale efficiently while controlling active compute.
### Data Pipeline and Scale
Generalization begins with data. LingBot-VLA 2.0 is pre-trained on roughly **60,000 hours** of data across **20 robot configurations**, spanning single-arm manipulators to full humanoids. The total raw pool exceeds **90,000 robot hours** and **20,000 egocentric video hours**. A multi-stage filtering pipeline removes noisy samples using smoothness metrics (jerk, velocity, acceleration), replay consistency checks, visual quality assessment, and egocentric SLAM/MANO hand-pose reconstruction.
Language models automatically annotate videos, segmenting them into temporally contiguous subtasks and assigning atomic actions from a closed vocabulary of 18 categories. Move and transit actions dominate the action distribution, providing strong priors for manipulation.
### Unified Action Representation
Because robots expose different joint layouts, LingBot-VLA 2.0 uses a **fixed 55-dimensional canonical vector** to represent both states and actions across all embodiments. This unified representation includes arm joint positions, end-effector poses (XYZ + quaternion), gripper position, hand joint positions, waist pose, head pose, mobility, and reserved dimensions. The design enables a single policy to control diverse platforms without retraining.
### MoE Action Expert and Predictive Dynamics
The action expert replaces standard feed-forward networks with sparse MoE layers. Guided by a sigmoid-based, auxiliary-loss-free routing strategy inspired by DeepSeek-V3, only the top-K experts activate per token. Despite sparse activation, the MoE model achieves lower training loss than a dense baseline and lower validation action error on GM-100 tasks.
For anticipation, LingBot-VLA 2.0 introduces **dual-query distillation**. Learnable queries are appended to visual and text tokens: one targeting the current observation, and another targeting a future observation T steps ahead. Two teachers supervise these queries—LingBot-Depth provides geometric cues, while DINO-Video offers temporally grounded semantic priors. DINO-Video, trained on 5 million video clips across internet, egocentric, and robotic data, leads on three of four LARYBench metrics.
### Benchmark Results
On the **GM-100** bimanual benchmark, a single LingBot-VLA 2.0 policy is jointly trained on nine tasks per embodiment. Compared to prior models, LingBot-VLA 2.0 consistently improves both progress score and success rate. Gains are especially pronounced on tasks requiring precise object grounding—for example, the AgileX Retrieve keychain task improves from 60% to 100% success over version 1.0. Some tasks still show a gap between progress and final success, pointing to challenges in the last precise placement or release steps.
On long-horizon mobile manipulation under in-domain and out-of-domain conditions, LingBot-VLA 2.0 outperforms π0.5 across both settings, with larger improvements in OOD scenarios where object pose and layout vary.
### Getting Started and Deployment
The repository includes install, download, and deployment scripts. Weights can be downloaded with a simple Python command, and deployed with compiled inference on real robots. Post-training leverages LeRobot datasets, with examples based on RoboTwin 2.0 across 50 tasks. The framework supports both sequence-wise auxiliary routing loss and a loss-free setup, with Muon and AdamW optimizers exposed.
### Use Cases and Examples
The expanded action space enables multi-modal, long-horizon manipulation:
– **Kitchen mobile manipulation:** Sorting fruits and drinks into a fridge with base navigation, door opening, and placement.
– **Surface cleaning:** Wiping foam off a stove while chaining grasping, wiping, and tool repositioning.
– **Bimanual packing and sorting:** Tasks such as egg packing, tool packing, and block sorting.
– **Dexterous-hand control:** Using 12-DoF hands on platforms like Unitree G1, Fourier GR-2, and AgiBot A2.
### Interactive Dynamic Explainer
An interactive explorer lets users inspect the unified 55-D action space, explore ablation studies, and review GM-100 benchmark results across models. The tool highlights design choices that drive LingBot-VLA 2.0’s performance gains, including relative action representation, MoE routing, normalization strategy, and objective choice.
### Conclusion
LingBot-VLA 2.0 represents a significant step toward deployable robot foundation models. By unifying action representations, scaling data and parameters with MoE, and introducing predictive dynamics via dual-query distillation, it delivers stronger generalization and robustness in real-world scenarios. The open-source release lowers the barrier for researchers and engineers to build on top of a strong, generalist robot policy.
**Original Article Source:**
