Runway's video AI can now pilot robots

AI company Runway has open-sourced its robot control model, Praxis-1, making both the model weights and source code freely available under a commercial-friendly license. The system uses camera images and text instructions to predict changes in a physical environment and calculate precise robot movements, building directly on Runway's expertise in video generation.

The model leverages pre-training on vast amounts of ordinary video footage to understand object motion, which could significantly reduce the need for expensive, robot-specific training datasets. Early testing partners have deployed it on various platforms, from industrial arms to humanoid robots. This release marks a major step in translating abstract video understanding into concrete, real-world physical control, potentially lowering the barrier to advanced robotics.