RoboLab-120 · Simulated Franka Panda
NVIDIA's 120 RoboLab tasks in Isaac Sim, on a simulated Franka Panda with a Robotiq 2F-85 gripper (NVIDIA RoboLab).
FLUX 3 Action
Put one (1) more bananas in the grey bin.
Success in 7 s of simulated time
π₀.₅
Make sure there are 3 (three) bananas in the grey bin.
Success in 7 s of simulated time
Cosmos3 Nano
Put 2 bananas in the crate
Success in 13 s of simulated time
GR00T
Pick up a fruit
Success in 5 s of simulated time
MolmoAct2
Put the red bbq sauce bottle in the crate
Success in 10 s of simulated time
G0.5
Put the mustard in the right bin
Success in 15 s of simulated time
| # | Model | Episodes | Reliability @ 300 s | Speed @ 300 s |
|---|---|---|---|---|
| % subtask score | tasks/min | |||
| Loading the benchmark… | ||||
Reliability is RoboLab's mean subtask score in simulated time, over 120 tasks × 3 episodes. ± is half the 95% range.
How reliable is it at each time limit?
The mean subtask score the episodes had reached within each time limit, with its 95% range over the tasks. An episode that has ended keeps its final score. Drag the marker to rank the table at another limit.
More charts
When do the successes come?
The success marks of each model's episodes, smoothed, as a share of its episodes per second of simulated time.
Is it faster, or more reliable?
Each line is a model at every time limit from 1 s to 300 s: its Speed in tasks/min across, its Reliability up. The dot is the current time limit, and the dashed lines are its 95% ranges. The axis ends at 6 tasks/min, so the quickest successes at a short limit fall beyond it.
Do two models really differ?
Episodes
Each episode and its own result, the newest first.