NVIDIA has released Cosmos 3 Edge, a 4-billion-parameter open world model built to run on-device. It helps robots and vision AI agents understand surroundings, reason in real time, and generate robot actions locally.
The Cosmos 3 family included Cosmos 3 Nano (16B) and Cosmos 3 Super (64B) shipped on May 31, 2026 at GTC Taipei. Edge is the third and smallest tier, at roughly one-sixteenth the size of Super.
The problem is specific. Machines operate at the edge in factories, warehouses, and hospitals. They need data center–level performance on memory-constrained systems. Cosmos 3 Edge targets that gap.
One action representation across embodiments
Physical systems describe actions differently. A vehicle uses ego pose and movement. A camera uses camera motion. A robot arm uses the pose of its end effector, and a gripper adds grasp state.
Cosmos 3 maps these embodiments into a common action representation. Actions are encoded as compact geometric vectors that capture translation, rotation, and manipulation state.
This connects control to the visual structure of the world. The model associates pixel changes with physical motion and control inputs. Generated video then becomes more than a prediction. It represents how the world should change in response to an action.
Supported action dimensions depend on the embodiment. The Cosmos GitHub repository lists camera motion (9D), autonomous vehicle (9D), egocentric motion (57D), single-arm robot (10D), dual-arm robot (20D), and humanoid robot (29D).
Policy mode runs in both directions
As a policy, Cosmos 3 Edge predicts an action together with its expected visual consequence. Current state goes in; an action and its likely visual outcome come out.
Action flows in both directions. The model can predict the effect of an action, or infer the action from its effect. This connects world modeling directly to robot policy training and evaluation.
NVIDIA also released Cosmos 3 Edge Policy (DROID). It is a robot manipulation policy post-trained on the DROID dataset for pick-and-place tasks, with post-training scripts included. Developers can fine-tune on a small H100 cluster or an NVIDIA DGX Station before deployment.
Is it Deployable
Cosmos 3 Edge delivers memory-efficient inference across NVIDIA edge computers. Targets include NVIDIA RTX PRO GPUs, NVIDIA DGX, GeForce RTX GPUs, and NVIDIA Jetson, including the newly announced Jetson T2000 and T3000 modules.
As a post-trained world action model (WAM), the model operates at robot-control resolution of 640×360 observations. On NVIDIA Jetson Thor it generates 32 actions per inference, while achieving real-time control at 15 Hz. For generation, the Edge tier supports 256p and 480p resolutions, 12–30 fps, and 50–150 frames.
Using the open Cosmos framework, developers can post-train Cosmos 3 Edge for a specific embodiment and sensor set in about a day. NVIDIA positions a GeForce RTX 3070 or better as a local on-ramp for prototyping.
Key Takeaways
- Cosmos 3 Edge is a 4B open world model (2B dense reasoner) that runs on-device, released July 20 on Hugging Face.
- A Mixture-of-Transformers design pairs an autoregressive reasoner tower with a diffusion generator tower through shared multimodal attention.
- It hits 640×360 control resolution, 32 actions per inference, and 15 Hz real-time control on NVIDIA Jetson Thor.
- Actions map to a common translation/rotation/manipulation representation, spanning camera, vehicle, single-arm, dual-arm, and humanoid embodiments.
- Benchmarks (#1 on VANTAGE-Bench at 4B) are internally claimed; the model ships under Linux Foundation OpenMDW-1.1.
Sources: Hugging Face launch post, Cosmos3-Edge model card, Cosmos 3 collection, NVIDIA technical report (PDF), Cosmos GitHub, NVIDIA developer blog, Jetson Thor blog and NVIDIA Newsroom: Japan coalition
The post NVIDIA Releases Cosmos 3 Edge: A 4B-Parameter Open World Model That Reasons and Generates Robot Actions On-Device appeared first on MarkTechPost.