## What does world model do here
A world model learns how an environment changes over time. It represents objects, motion, spatial relationships, and the effects of actions.
Consider a robot reaching for an object. Recognizing the object is only the first step. The robot must also track where the object is, how its gripper moves, and what happens on contact. A world model reasons about these relationships. It can predict the visual result of an action, infer the action that caused a change, or generate an action to reach a goal.
Cosmos 3 Edge brings these capabilities into one on-device model. Its shared representation lets a system understand the current world state, simulate possible futures, and connect those futures to actions.
## Two transformer towers, one shared representation
Cosmos 3 uses a Mixture-of-Transformers architecture with two towers:
– The **autoregressive tower** processes vision and text tokens for understanding and reasoning.
– The **diffusion tower** processes vision, audio, and action tokens for prediction, generation, and neural simulation.
Each tower keeps separate normalization layers and multilayer perceptrons, while they share multimodal attention layers that align information across language, video, audio, and action. This setup lets the model reason about a scene before generating an output. Cosmos 3 Edge uses a 2B dense transformer for its reasoner and follows Qwen3-VL-compatible message conventions for image and video inputs.
## One action representation across embodiments
Physical systems describe actions differently. A vehicle uses ego pose and movement, a camera uses camera motion, a robot arm uses the pose of its end effector, and a gripper adds grasp state.
Cosmos 3 maps these embodiments into a common action representation. Actions are encoded as compact geometric vectors that capture translation, rotation, and manipulation state. This connects control to the visual structure of the world. The model associates pixel changes with physical motion and control inputs. Generated video then becomes more than a prediction; it represents how the world should change in response to an action.
Supported action dimensions depend on the embodiment: camera motion (9D), autonomous vehicle (9D), egocentric motion (57D), single-arm robot (10D), dual-arm robot (20D), and humanoid robot (29D).
## Policy mode runs in both directions
As a policy, Cosmos 3 Edge predicts an action together with its expected visual consequence. Current state goes in; an action and its likely visual outcome come out.
Action flows in both directions. The model can predict the effect of an action or infer the action from its effect. This connects world modeling directly to robot policy training and evaluation.
NVIDIA also released Cosmos 3 Edge Policy (DROID), a robot manipulation policy post-trained on the DROID dataset for pick-and-place tasks, with post-training scripts included. Developers can fine-tune on a small H100 cluster or an NVIDIA DGX Station before deployment.
## Is it deployable
Cosmos 3 Edge delivers memory-efficient inference across NVIDIA edge computers. Targets include NVIDIA RTX PRO GPUs, NVIDIA DGX, GeForce RTX GPUs, and NVIDIA Jetson, including the newly announced Jetson T2000 and T3000 modules.
As a post-trained world action model (WAM), the model operates at robot-control resolution of 640×360 observations. On NVIDIA Jetson Thor it generates 32 actions per inference, while achieving real-time control at 15 Hz. For generation, the Edge tier supports 256p and 480p resolutions, 12–30 fps, and 50–150 frames.
Using the open Cosmos framework, developers can post-train Cosmos 3 Edge for a specific embodiment and sensor set in about a day. NVIDIA positions a GeForce RTX 3070 or better as a local on-ramp for prototyping.
# FAQ
What is a world model, and why does Cosmos 3 Edge need one?
A world model learns how an environment changes over time and represents objects, motion, spatial relationships, and the effects of actions. Cosmos 3 Edge uses it to understand the current world state, simulate possible futures, and connect those futures to robot actions.
How many parameters does Cosmos 3 Edge have, and what architecture does it use?
Cosmos 3 Edge has about 4B total parameters (2B dense) and uses a Mixture-of-Transformers with two towers: an autoregressive tower for reasoning and a diffusion tower for prediction and generation, with shared multimodal attention layers.
What action representations does Cosmos 3 Edge support?
It supports multiple embodiments: camera motion (9D), autonomous vehicle (9D), egocentric motion (57D), single-arm robot (10D), dual-arm robot (20D), and humanoid robot (29D). Actions are encoded as compact geometric vectors.
Can Cosmos 3 Edge run on edge devices?
Yes. It is designed for memory-efficient inference on NVIDIA edge hardware such as Jetson T2000, T3000, Jetson AGX Thor, and also works on RTX PRO and GeForce RTX GPUs for local prototyping.
What policy modes does Cosmos 3 Edge offer?
It can predict actions from the current state (forward mode) and infer actions from observed effects (inverse mode), enabling robot policy training and evaluation. Cosmos 3 Edge Policy (DROID) is available for pick-and-place tasks and can be post-trained for specific embodiments.
# Conclusion
Cosmos 3 Edge brings a on-device world model that combines two transformer towers into a single shared representation, enabling reasoning and generation across vision, text, audio, and action. With a compact action representation that spans multiple embodiments and policy modes that work in both forward and inverse directions, it offers a flexible foundation for robot learning and control. Deployable across NVIDIA’s edge and workstation hardware, Cosmos 3 Edge lowers the barrier for prototyping and deploying sophisticated robot policies in real-world environments.
