PhAIL
PhAIL Physical AI Leaderboard by Positronic Robotics
Leaderboard › Model
Generalist VLA Finetuning

Model card

Made by
Physical Intelligence
Size
3B parameters
Published
22 Apr 2025
More about the model
Publication
arXiv 2504.16054
Code
github.com/Physical-Intelligence/openpi
Weights licence
Apache 2.0, with the Gemma Terms of Use source
Actions per chunk
π₀.₅: 50 source; π₀.₅ DROID: 15 source
Smallest GPU
More than 8 GB of GPU memory, for example an RTX 4090 source
Inference time
About 76 ms per chunk on an RTX 4090 (bfloat16, 5 denoising steps) source
Architecture
PaliGemma 2B vision-language model and a 300M flow-matching action expert source
Training data
Base model: 10k+ hours of robot data. π₀.₅ DROID: fine-tuned from it on the DROID dataset source

Run with Positronic

π₀.₅ DROID

Start the server source

docker run --gpus all -p 8000:8000 positro/pi05-droid

Drive a DROID robot from a positronic checkout guide

uv run --extra hardware positronic-inference real \
  --policy=.remote \
  --policy.address.host=localhost --policy.address.port=8000 \
  --next_task.instruction="Put the marker on the plate"

Progress through an episode

Each band shows how many episodes of π₀.₅ had succeeded, partly succeeded (some subtask score) or had no success at each second of simulated time. An episode that has ended stays in its band. The dashed line shows the mean Reliability, the mean subtask score.

Episodes

Each episode of π₀.₅ on RoboLab-120, the newest first.

360 episodes
Task Task difficulty Result Subtask score, % Time to success, s
Pick up the banana and put it on the plate Easy Success 100 10.4
Pick up the banana and put it on the plate Easy Success 100 10.5
Pick up the banana and put it on the plate Easy Success 100 12.1
Make sure there are 3 (three) bananas in the grey bin. Easy Success 100 8.6
Make sure there are 3 (three) bananas in the grey bin. Easy Success 100 7.1
Make sure there are 3 (three) bananas in the grey bin. Easy Success 100 8.1
Place the larger object in the grey bin. Very hard Fail 0 -
Place the larger object in the grey bin. Very hard Fail 0 -
Place the larger object in the grey bin. Very hard Fail 0 -
Pick up a fruit Easy Fail 100 -
Pick up a fruit Easy Fail 100 -
Pick up a fruit Easy Fail 100 -
Put the cube then the banana in the bowl Medium Fail (1/2) 50 -
Put the cube then the banana in the bowl Medium Fail (0/2) 0 -
Put the cube then the banana in the bowl Medium Fail (0/2) 0 -
Stack the left bowl on the right bowl Hard Success 100 17.2
Stack the left bowl on the right bowl Hard Fail 0 -
Stack the left bowl on the right bowl Hard Success 100 14.6
Put the red bbq sauce bottle in the crate Easy Fail 0 -
Put the red bbq sauce bottle in the crate Easy Success 100 24.9
Put the red bbq sauce bottle in the crate Easy Success 100 29.7
Make sure all the white mugs are upright so that the opening is facing upwards. Hard Fail 0 -
Make sure all the white mugs are upright so that the opening is facing upwards. Hard Success 100 35.2
Make sure all the white mugs are upright so that the opening is facing upwards. Hard Success 100 13.4
Put all the oranges on the plate Easy Success 100 8.8
Put all the oranges on the plate Easy Success 100 10.9
Put all the oranges on the plate Easy Fail 50 -
Stack the white mugs on top of each other. Hard Fail 0 -
Stack the white mugs on top of each other. Hard Fail 0 -
Stack the white mugs on top of each other. Hard Fail 0 -
Pack canned foods into the bin Hard Success 100 30.3
Pack canned foods into the bin Hard Fail 0 -
Pack canned foods into the bin Hard Fail 0 -
Put the smartphone in the grey bin Very hard Fail 0 -
Put the smartphone in the grey bin Very hard Fail 0 -
Put the smartphone in the grey bin Very hard Fail 0 -
Place the mustard on the raisin box. Very hard Success 0 9.7
Place the mustard on the raisin box. Very hard Success 0 9.7
Place the mustard on the raisin box. Very hard Success 0 9.1
Put 2 bananas in the crate Easy Success 100 9.3
Put 2 bananas in the crate Easy Success 100 30.8
Put 2 bananas in the crate Easy Success 100 39.9
Pick up the banana and place it in the bowl Easy Success 100 26.3
Pick up the banana and place it in the bowl Easy Success 100 15.7
Pick up the banana and place it in the bowl Easy Success 100 12.1
Put any white plastic bottle in the square pail Easy Success 100 13.8
Put any white plastic bottle in the square pail Easy Success 100 12.9
Put any white plastic bottle in the square pail Easy Success 100 11.1
Put the cube or the banana in the bowl Easy Success 100 8.1
Put the cube or the banana in the bowl Easy Fail 0 -
Put the cube or the banana in the bowl Easy Success 100 8.7
Put the cube in the bowl Easy Success 100 18.0
Put the cube in the bowl Easy Success 100 8.4
Put the cube in the bowl Easy Success 100 10.5
Stack the right bowl on the left bowl Hard Fail 0 -
Stack the right bowl on the left bowl Hard Success 100 17.9
Stack the right bowl on the left bowl Hard Fail 0 -
Place the smaller object in the grey bin. Medium Success 100 16.5
Place the smaller object in the grey bin. Medium Success 100 9.3
Place the smaller object in the grey bin. Medium Fail 0 -
put the bowl in the grey bin Easy Fail 0 -
put the bowl in the grey bin Easy Success 100 43.4
put the bowl in the grey bin Easy Success 100 21.3
Put the white mug in the center of the table. Easy Fail 100 -
Put the white mug in the center of the table. Easy Fail 100 -
Put the white mug in the center of the table. Easy Success 100 14.7
Put the cube and the banana in the bowl Easy Fail 50 -
Put the cube and the banana in the bowl Easy Success 100 21.5
Put the cube and the banana in the bowl Easy Fail 50 -
Put the mustard in the right bin Easy Success 100 11.8
Put the mustard in the right bin Easy Success 100 18.3
Put the mustard in the right bin Easy Fail 0 -
Pick up the orange measuring cup Medium Fail 50 -
Pick up the orange measuring cup Medium Fail 50 -
Pick up the orange measuring cup Medium Fail 50 -
Put the mustard in the left bin Medium Fail 0 -
Put the mustard in the left bin Medium Fail 0 -
Put the mustard in the left bin Medium Fail 0 -
Put the rubiks cube to the left of the bowl Easy Fail 67 -
Put the rubiks cube to the left of the bowl Easy Fail 67 -
Put the rubiks cube to the left of the bowl Easy Fail 67 -
Pick up the eye glasses Medium Fail 50 -
Pick up the eye glasses Medium Success 100 11.0
Pick up the eye glasses Medium Fail 50 -
put the banana then the cube in the bowl Medium Fail (2/2) 100 -
put the banana then the cube in the bowl Medium Success 100 25.2
put the banana then the cube in the bowl Medium Fail (1/2) 50 -
Take the white colored measuring spoon out of the red bowl and put it on the table. Easy Fail 0 -
Take the white colored measuring spoon out of the red bowl and put it on the table. Easy Fail 0 -
Take the white colored measuring spoon out of the red bowl and put it on the table. Easy Fail 50 -
Put one (1) more bananas in the grey bin. Easy Success 100 41.4
Put one (1) more bananas in the grey bin. Easy Success 100 7.6
Put one (1) more bananas in the grey bin. Easy Success 100 8.1
Grab a bagel Medium Fail 0 -
Grab a bagel Medium Fail 0 -
Grab a bagel Medium Fail 0 -
Stack the yellow block on the red block Medium Success 100 24.5
Stack the yellow block on the red block Medium Success 100 26.1
Stack the yellow block on the red block Medium Fail 50 -
Put the small red yogurt in the red bowl Very hard Fail 0 -
Put the small red yogurt in the red bowl Very hard Fail 0 -
Put the small red yogurt in the red bowl Very hard Success 100 23.6
Throw away the apple Very hard Fail 0 -
Throw away the apple Very hard Fail 0 -
Throw away the apple Very hard Fail 0 -
Pick up the butter box and place it on top of the raisin box Very hard Success 0 18.5
Pick up the butter box and place it on top of the raisin box Very hard Fail 0 -
Pick up the butter box and place it on top of the raisin box Very hard Success 0 34.0
Pick up the cordless drill. Easy Fail 100 -
Pick up the cordless drill. Easy Fail 100 -
Pick up the cordless drill. Easy Fail 0 -
Take the bananas out Easy Fail 100 -
Take the bananas out Easy Success 100 26.3
Take the bananas out Easy Success 100 26.3
Pack boxed foods into the bin Very hard Fail 0 -
Pack boxed foods into the bin Very hard Fail 0 -
Pack boxed foods into the bin Very hard Fail 0 -
Pick up the large blue pitcher Medium Fail 50 -
Pick up the large blue pitcher Medium Fail 50 -
Pick up the large blue pitcher Medium Fail 50 -
Put any white plastic bottle on the shelf Very hard Fail 0 -
Put any white plastic bottle on the shelf Very hard Fail 0 -
Put any white plastic bottle on the shelf Very hard Fail 0 -
Put the rubiks cube in front of the bowl Medium Fail 33 -
Put the rubiks cube in front of the bowl Medium Fail 33 -
Put the rubiks cube in front of the bowl Medium Fail 67 -
Put the rubiks cube to the right of the bowl Easy Fail 67 -
Put the rubiks cube to the right of the bowl Easy Fail 67 -
Put the rubiks cube to the right of the bowl Easy Fail 67 -
Put the red dishware in the grey bin Easy Fail 0 -
Put the red dishware in the grey bin Easy Success 100 48.9
Put the red dishware in the grey bin Easy Fail 50 -
Put the serving bowl anywhere on the shelf in front of you Very hard Fail 0 -
Put the serving bowl anywhere on the shelf in front of you Very hard Fail 0 -
Put the serving bowl anywhere on the shelf in front of you Very hard Fail 0 -
Put the rubiks cube behind the bowl Easy Success 100 16.1
Put the rubiks cube behind the bowl Easy Fail 67 -
Put the rubiks cube behind the bowl Easy Success 100 14.9
Stack the rubiks cubes in a tower Hard Success 100 33.1
Stack the rubiks cubes in a tower Hard Fail (1/2) 50 -
Stack the rubiks cubes in a tower Hard Success 100 30.9
Put the computer mouse on the keyboard Hard Success 100 30.8
Put the computer mouse on the keyboard Hard Success 100 33.4
Put the computer mouse on the keyboard Hard Success 100 24.0
Pick up the green vegetable block Hard Fail 50 -
Pick up the green vegetable block Hard Fail 50 -
Pick up the green vegetable block Hard Fail 50 -
Put the bagels on the plate Very hard Fail 0 -
Put the bagels on the plate Very hard Fail 0 -
Put the bagels on the plate Very hard Fail 0 -
Put three (3) fruits on the plate Easy Fail 67 -
Put three (3) fruits on the plate Easy Success 100 34.8
Put three (3) fruits on the plate Easy Fail 67 -
Put the canned food in the grey bin Hard Success 100 12.3
Put the canned food in the grey bin Hard Fail 0 -
Put the canned food in the grey bin Hard Success 100 47.1
Take the keyboard out of the bin and put it on the table Hard Fail 0 -
Take the keyboard out of the bin and put it on the table Hard Fail 0 -
Take the keyboard out of the bin and put it on the table Hard Fail 0 -
Put the red mug upright so that the opening is facing upwards. Hard Success 100 40.0
Put the red mug upright so that the opening is facing upwards. Hard Fail 0 -
Put the red mug upright so that the opening is facing upwards. Hard Fail 0 -
Take the spatula off the shelf and put it on the table Very hard Fail 0 -
Take the spatula off the shelf and put it on the table Very hard Fail 0 -
Take the spatula off the shelf and put it on the table Very hard Fail 0 -
Put the pink spaghetti spoon in the pot Hard Fail 0 -
Put the pink spaghetti spoon in the pot Hard Fail 0 -
Put the pink spaghetti spoon in the pot Hard Success 100 49.9
Put the bigger pumpkin in the bin Hard Success 100 30.7
Put the bigger pumpkin in the bin Hard Fail 0 -
Put the bigger pumpkin in the bin Hard Success 100 36.0
Put the onion in the wood bowl Easy Fail 0 -
Put the onion in the wood bowl Easy Fail 0 -
Put the onion in the wood bowl Easy Fail 0 -
Put all the red things in the grey bin Medium Success 100 46.0
Put all the red things in the grey bin Medium Fail 50 -
Put all the red things in the grey bin Medium Fail 50 -
Put the spring clamp in the right bin Hard Fail 0 -
Put the spring clamp in the right bin Hard Fail 0 -
Put the spring clamp in the right bin Hard Fail 0 -
Select the cordless drill and put it on the table Very hard Fail 0 -
Select the cordless drill and put it on the table Very hard Fail 0 -
Select the cordless drill and put it on the table Very hard Fail 0 -
Put all the small plastic bottles in the square pail Easy Fail 67 -
Put all the small plastic bottles in the square pail Easy Success 100 92.4
Put all the small plastic bottles in the square pail Easy Success 100 62.5
Move the pink tool from this utensil container to the other utensil holder Very hard Fail 0 -
Move the pink tool from this utensil container to the other utensil holder Very hard Fail 0 -
Move the pink tool from this utensil container to the other utensil holder Very hard Fail 0 -
Clean up the white mugs Very hard Fail 0 -
Clean up the white mugs Very hard Fail 0 -
Clean up the white mugs Very hard Fail 0 -
Stand the jug upright Very hard Fail 0 -
Stand the jug upright Very hard Fail 0 -
Stand the jug upright Very hard Fail 0 -
Put the whiteboard marker in the mug Very hard Fail 0 -
Put the whiteboard marker in the mug Very hard Fail 0 -
Put the whiteboard marker in the mug Very hard Fail 0 -
Put the metal spoon that's in the wooden bowl in the mug Medium Fail 0 -
Put the metal spoon that's in the wooden bowl in the mug Medium Success 100 24.1
Put the metal spoon that's in the wooden bowl in the mug Medium Success 100 43.5
Stack the blocks into a tower Medium Fail (2/3) 66 -
Stack the blocks into a tower Medium Success 100 84.3
Stack the blocks into a tower Medium Fail (2/3) 66 -
Move an orange or a lime to the wood bowl Medium Fail 0 -
Move an orange or a lime to the wood bowl Medium Success 100 8.5
Move an orange or a lime to the wood bowl Medium Fail 0 -
Select the blue hammer and put it on the table Hard Fail 100 -
Select the blue hammer and put it on the table Hard Fail 0 -
Select the blue hammer and put it on the table Hard Fail 0 -
Sort all rubiks cubes into the bin Easy Fail 67 -
Sort all rubiks cubes into the bin Easy Success 100 103.3
Sort all rubiks cubes into the bin Easy Success 100 62.6
Put all white objects and yellow objects in the grey bin Very hard Fail 50 -
Put all white objects and yellow objects in the grey bin Very hard Fail 50 -
Put all white objects and yellow objects in the grey bin Very hard Fail 50 -
Put the lizard away in the bin Very hard Fail 0 -
Put the lizard away in the bin Very hard Fail 0 -
Put the lizard away in the bin Very hard Fail 0 -
Put the lizards in the bin Hard Fail 0 -
Put the lizards in the bin Hard Fail 0 -
Put the lizards in the bin Hard Fail 0 -
Stack the blocks in the order from bottom to top: red, blue, green, yellow Very hard Fail (0/3) 0 -
Stack the blocks in the order from bottom to top: red, blue, green, yellow Very hard Fail (0/3) 0 -
Stack the blocks in the order from bottom to top: red, blue, green, yellow Very hard Fail (0/3) 0 -
Put the onion on the plate Medium Success 100 15.1
Put the onion on the plate Medium Success 100 14.0
Put the onion on the plate Medium Fail 0 -
Put the coffee pot in the grey bin Very hard Fail 0 -
Put the coffee pot in the grey bin Very hard Fail 0 -
Put the coffee pot in the grey bin Very hard Fail 0 -
Put the phone or the remote in the grey bin Hard Fail 0 -
Put the phone or the remote in the grey bin Hard Fail 0 -
Put the phone or the remote in the grey bin Hard Fail 0 -
Move an orange to the white bowl Very hard Fail 0 -
Move an orange to the white bowl Very hard Fail 0 -
Move an orange to the white bowl Very hard Fail 0 -
Put hammers in the right bin and do not touch anything else Very hard Fail 50 -
Put hammers in the right bin and do not touch anything else Very hard Fail 50 -
Put hammers in the right bin and do not touch anything else Very hard Success 100 76.6
Put the recyclable cartons in the grey bin Very hard Fail 50 -
Put the recyclable cartons in the grey bin Very hard Fail 50 -
Put the recyclable cartons in the grey bin Very hard Fail 50 -
Put the cartons that can be recycled in the vertical crate Very hard Fail 50 -
Put the cartons that can be recycled in the vertical crate Very hard Fail 50 -
Put the cartons that can be recycled in the vertical crate Very hard Fail 50 -
Put the cartons that can be recycled on the box Very hard Fail 0 -
Put the cartons that can be recycled on the box Very hard Fail 0 -
Put the cartons that can be recycled on the box Very hard Fail 0 -
Unstack the rubiks cube tower Easy Success 100 53.1
Unstack the rubiks cube tower Easy Success 100 56.0
Unstack the rubiks cube tower Easy Success 100 28.9
Reorient all the mugs upright so that the opening is facing upwards. Hard Fail 67 -
Reorient all the mugs upright so that the opening is facing upwards. Hard Fail 67 -
Reorient all the mugs upright so that the opening is facing upwards. Hard Fail 0 -
Put the wooden spatula in the bowl Hard Success 100 19.0
Put the wooden spatula in the bowl Hard Fail 0 -
Put the wooden spatula in the bowl Hard Fail 0 -
Put the small pumpkin in the bin Very hard Fail 0 -
Put the small pumpkin in the bin Very hard Fail 0 -
Put the small pumpkin in the bin Very hard Fail 0 -
Put the red BBQ sauce bottles in the grey bin Hard Fail 50 -
Put the red BBQ sauce bottles in the grey bin Hard Fail 0 -
Put the red BBQ sauce bottles in the grey bin Hard Fail 50 -
Put all the jugs on the shelf Very hard Fail 0 -
Put all the jugs on the shelf Very hard Fail 0 -
Put all the jugs on the shelf Very hard Fail 0 -
Put the two measuring cups outside of the plate Hard Success 50 57.3
Put the two measuring cups outside of the plate Hard Success 50 47.8
Put the two measuring cups outside of the plate Hard Success 50 31.2
Put all the pumpkins away in the bin Very hard Fail 50 -
Put all the pumpkins away in the bin Very hard Fail 50 -
Put all the pumpkins away in the bin Very hard Fail 0 -
Pack canned foods into the bin Hard Fail 0 -
Pack canned foods into the bin Hard Fail 50 -
Pack canned foods into the bin Hard Success 100 167.4
Put all the green fruit on the plate Easy Success 100 61.2
Put all the green fruit on the plate Easy Fail 50 -
Put all the green fruit on the plate Easy Fail 50 -
Pack yellow objects in right container and blue object in the left container Very hard Fail (0/2) 0 -
Pack yellow objects in right container and blue object in the left container Very hard Fail (0/2) 0 -
Pack yellow objects in right container and blue object in the left container Very hard Fail (0/2) 0 -
Put the black items in the grey bin Hard Fail 20 -
Put the black items in the grey bin Hard Fail 40 -
Put the black items in the grey bin Hard Fail 20 -
Move the bananas to the bagel plate Hard Fail 0 -
Move the bananas to the bagel plate Hard Fail 0 -
Move the bananas to the bagel plate Hard Fail 0 -
Put away the snacks in the bin Very hard Fail 0 -
Put away the snacks in the bin Very hard Fail 0 -
Put away the snacks in the bin Very hard Fail 0 -
Put the fork and spoon in the ceramicmug Very hard Fail 0 -
Put the fork and spoon in the ceramicmug Very hard Fail 0 -
Put the fork and spoon in the ceramicmug Very hard Fail 0 -
Take the mugs off the shelf Easy Fail 0 -
Take the mugs off the shelf Easy Fail 50 -
Take the mugs off the shelf Easy Fail 50 -
Put the non-hammer tools in the right bin Very hard Fail 0 -
Put the non-hammer tools in the right bin Very hard Fail 0 -
Put the non-hammer tools in the right bin Very hard Fail 0 -
Put the apple and yogurt in the bowl Hard Fail 50 -
Put the apple and yogurt in the bowl Hard Success 100 113.7
Put the apple and yogurt in the bowl Hard Fail 50 -
Put the two mugs on the shelf Hard Fail 0 -
Put the two mugs on the shelf Hard Fail 50 -
Put the two mugs on the shelf Hard Fail 50 -
Put the red hammer and black hammer in the left bin Very hard Fail 0 -
Put the red hammer and black hammer in the left bin Very hard Fail 50 -
Put the red hammer and black hammer in the left bin Very hard Fail 50 -
Pack boxed foods into the bin Very hard Fail 0 -
Pack boxed foods into the bin Very hard Fail 50 -
Pack boxed foods into the bin Very hard Fail 0 -
Put the red hammer and black hammer in the left bin Very hard Fail 50 -
Put the red hammer and black hammer in the left bin Very hard Fail 100 -
Put the red hammer and black hammer in the left bin Very hard Fail 0 -
Put all the cubes and blocks in the bin Easy Success 100 210.3
Put all the cubes and blocks in the bin Easy Success 100 135.3
Put all the cubes and blocks in the bin Easy Success 100 213.7
Sort all colored blocks into the bin Medium Success 100 135.3
Sort all colored blocks into the bin Medium Fail 50 -
Sort all colored blocks into the bin Medium Fail 75 -
Put the dishware in the grey bin Medium Fail 33 -
Put the dishware in the grey bin Medium Fail 33 -
Put the dishware in the grey bin Medium Fail 33 -
Put two (2) mugs on the wire shelf Hard Fail 0 -
Put two (2) mugs on the wire shelf Hard Fail 100 -
Put two (2) mugs on the wire shelf Hard Fail 50 -
Pack canned foods into the bin Hard Fail 67 -
Pack canned foods into the bin Hard Fail 33 -
Pack canned foods into the bin Hard Fail 33 -
Put all of the serving spoons with no holes in the pot Hard Fail 33 -
Put all of the serving spoons with no holes in the pot Hard Fail 0 -
Put all of the serving spoons with no holes in the pot Hard Fail 33 -
Sort the sauce condiments into the grey bin Hard Fail 0 -
Sort the sauce condiments into the grey bin Hard Fail 25 -
Sort the sauce condiments into the grey bin Hard Fail 0 -
Put the green spoons in the pot Hard Fail 0 -
Put the green spoons in the pot Hard Fail 33 -
Put the green spoons in the pot Hard Fail 33 -
Put the electronic devices in the grey bin Hard Fail 25 -
Put the electronic devices in the grey bin Hard Fail 25 -
Put the electronic devices in the grey bin Hard Fail 25 -
Put all plastic bottles away in the bin Medium Success 100 76.1
Put all plastic bottles away in the bin Medium Fail 67 -
Put all plastic bottles away in the bin Medium Fail 67 -
Pack boxed foods into the bin Very hard Fail 0 -
Pack boxed foods into the bin Very hard Fail 0 -
Pack boxed foods into the bin Very hard Fail 0 -
Take out all the hammers and put them on the table Very hard Fail 0 -
Take out all the hammers and put them on the table Very hard Fail 0 -
Take out all the hammers and put them on the table Very hard Fail 0 -
Clean up all the smaller toys and leave the birdhouse on the table Very hard Fail 0 -
Clean up all the smaller toys and leave the birdhouse on the table Very hard Fail 0 -
Clean up all the smaller toys and leave the birdhouse on the table Very hard Fail 0 -
Clear away the organic objects Very hard Fail 0 -
Clear away the organic objects Very hard Fail 0 -
Clear away the organic objects Very hard Fail 0 -
Put all the fruits on the plate Hard Fail 57 -
Put all the fruits on the plate Hard Fail 57 -
Put all the fruits on the plate Hard Fail 57 -