MolmoSpaces-320 · Simulated Franka FR3
320 tasks of Ai2's MolmoSpaces benchmarks in MuJoCo, on a simulated Franka FR3 with a Robotiq 2F-85 gripper (Ai2 MolmoSpaces).
FLUX 3 Action
close the drawer.
Success in 1 s of simulated time
Cosmos3 Nano
close the table.
Success in 3 s of simulated time
MolmoAct2
close the dresser.
Success in 1 s of simulated time
GR00T
close the white dresser.
Success in 1 s of simulated time
G0.5
close the cabinet.
Success in 5 s of simulated time
π₀.₅
pick up the tissue.
Success in 5 s of simulated time
| # | Model | Episodes | Reliability @ 46 s | Speed @ 46 s |
|---|---|---|---|---|
| % success | tasks/min | |||
| Loading the benchmark… | ||||
Reliability is the share of episodes that pass MolmoSpaces' own success check in simulated time, over 320 benchmark episodes × 1 episode. ± is half the 95% range.
How reliable is it at each time limit?
The share of the episodes that had succeeded within each time limit, with its 95% range over the benchmark episodes. Drag the marker to rank the table at another limit.
More charts
When do the successes come?
The success marks of each model's episodes, smoothed, as a share of its episodes per second of simulated time.
Is it faster, or more reliable?
Each line is a model at every time limit from 1 s to 46 s: its Speed in tasks/min across, its Reliability up. The dot is the current time limit, and the dashed lines are its 95% ranges. The axis ends at 8 tasks/min, so the quickest successes at a short limit fall beyond it.
Do two models really differ?
Episodes
Each episode and its own result, the newest first.