Berkeley researchers release code for AI agents to build and improve robot programs

AI agents can assemble robot-control programs, rehearse them in simulation and revise them after physical failures, according to UC Berkeley researchers. The team is presenting months of testing at the Conference on Robot Learning during the week of November 9, 2026, alongside open-source code and an industrial-task benchmark intended to help other roboticists try the approach.

Berkeley roboticist Ken Goldberg, NVIDIA robotics engineering director Spencer Huang and their collaborators call the approach “agentic robotics.” Their aim is to reduce two burdens: the engineering needed to hand-program robot behavior and the large datasets needed to teach it through demonstrations. Rather than choosing between those routes, the team uses AI coding agents to connect existing robot skills into programs that human engineers can inspect and refine.

The underlying Graph-as-Policy framework, or GaP, represents a task as connected steps for perception, planning and control. Those steps draw from a modular library: finding an object, planning a grasp or moving a gripper without a collision. Agents generate the connections, then use NVIDIA’s Isaac simulation libraries to rehearse variations, locate failures and change the program before it reaches a physical robot.

The research targets industrial work with variation in objects’ shapes and positions, rather than only fixed, identical movements. The paper’s authors report evaluating GaP on eight new benchmarks, split evenly between simulation and real-world tasks. They report success rates above their comparison systems; that finding concerns those evaluated tasks, not a demonstration of general-purpose robot competence.

One demonstration began with a short video of a robot wiping ink from a metal bar with a sponge. Goldberg said postdoctoral researchers Eric Kaiyuan Chen and Shuangyu Xie gave OpenAI’s Astra model that video and access to their simulation interface. In less than an hour, the model created and refined a physics simulation that reproduced the scene, including the sponge’s deformation, according to Berkeley’s account.

Using that simulation and GaP, the AI generated a control program, tested it repeatedly and improved it. The resulting code worked on a real robot. But one trial also exposed a bad assumption: the robot pushed the bar over because it had not reliably inferred its mass. Goldberg said video of that failure could feed back into the simulation, allowing the agent to revise and retest its program.

An earlier deployment provides practical context. In a September 17 account, Ambi Robotics described using a GaP-based system to address packages that failed to fit through sorting-bag openings. Sensor noise and package deformation could produce poor placement estimates, forcing retries. The company said agents examined 500 production events and generated and evaluated three proposed solutions within 10 hours, versus its estimate of several weeks for engineers.

−374Proposal 1

Ambi reported a small capacity loss in its production comparison.

−1,497Proposal 2

The second proposal also reduced capacity against the baseline.

+15,725Proposal 3

The successful proposal’s annual estimate came from 10 days of production A/B testing.

The winning change adjusted placement planning for dangling bags and flat items. Ambi measured an additional 4.2 packages per hour, then estimated annual capacity using a 12-hour day and six-day week—not a full year of observed gains. The company said the update reached 30% of its U.S. fleet. Its system also constrains agent edits to preserve safety checks and error handling, keeping the faster programming loop inside defined engineering boundaries.