GPT-6 Astra Put the cube or the banana in the bowl Success in 7 s of simulated time FLUX 3 Action Put one (1) more bananas in the grey bin. Success in 6 s of simulated time Cosmos3 Nano Pick up the orange measuring cup Success in 8 s of simulated time π₀.₅ Place the mustard on the raisin box. Success in 6 s of simulated time GR00T Pick up a fruit Success in 5 s of simulated time MolmoAct2 Put the red bbq sauce bottle in the crate Success in 6 s of simulated time G0.5 Make sure there are 3 (three) bananas in the grey bin. Success in 7 s of simulated time
Thinking is free

Turn this off to keep the world running while the model thinks, as on a real robot.

# Model Episodes Reliability @ 300 s Speed @ 300 s
% subtask score tasks/min
Loading the benchmark…

Reliability is RoboLab's mean subtask score in simulated time, over 120 tasks × 3 episodes. ± is half the 95% range.

Thinking is free: the simulation stops while the model thinks, so slow inference costs no time.

How reliable is it at each time limit?

The mean subtask score the episodes had reached within each time limit, with its 95% range over the tasks. An episode that has ended keeps its final score. Drag the marker to rank the table at another limit.

More charts

Episodes

Each episode and its own result, the newest first.