Egocentric video vs robot training data
Buy human video for pretraining. Buy robot data to train actions on your robot. Drag the line to compare.


A hand is not a gripper.
A hand, a gripper and a robot arm differ in joints, reach and speed. A head camera sees from eye level, a wrist camera sees the gripper. So human video needs mapping before a robot can learn actions from it.[1]
The scale differs too. EgoDex holds 829 hours of human video across 194 tasks.[2] DROID holds 350 hours of robot data, 76,000 trajectories across 86 tasks.[3]
Six data types, and which hold robot actions.
Check the type before you ask for a quote. Each card opens its suppliers.
Robot teleoperation
Records robot actionsUse Policy training on that robot.
Watch Tied to one robot.
Egocentric video
No robot actionsUse Pretraining and world models.
Watch Needs tracking and alignment first.
handhandTracked human demos
Actions inferredUse Mapping human motion to a robot.
Watch Pose points are estimates.
Gripper demos
Actions derivedUse Grasp and reach for a matching gripper.
Watch The gripper must match yours.
Labels
Labels onlyUse Finding and checking episodes.
Watch Labels are not actions by default.
Simulation
Actions in simUse Pretraining and evaluation.
Watch Realistic looks do not prove the physics.
Step 4 turns human video into robot data.
If the mapping to your robot fails, the episode stays a video.
Record
A person does the task while the camera runs.
Align
Sensor clocks are synced and the camera calibrated.
Track
Hand, body and head pose are estimated from video.
Map to the robot
Human pose becomes the robot’s joint or gripper space. This needs knowledge of the robot.
If this fails, the episode stays a videoCheck and label
Start, end, success and gaps are checked.
Write the format
Saved as LeRobot, RLDS or similar.
Train and test
Tested on held-out tasks on the real robot.
Put these 11 lines in your spec.
A supplier can only price what you define.
Test 5 to 10 episodes before you buy.
- Load them. Use the reader your training code uses. Pass or fail.
- Check each episode. A start, an end, an action column and rising timestamps.
- Check sync. Frames and robot state line up, and the calibration file is there.
- Check fps and resolution. Against your spec.
- Read the rights pack. Consent wording and a checksum per file.
- Count failures. Compare them with any rejection rate the supplier states.
- Pay for the sample. Keep the results with the order.
Know what you are buying.
Send one spec. We check samples against it before you order.
Nothing goes to a supplier without your yes.

Questions
Can I train a robot on egocentric video alone?
For actions on a specific robot, usually not. Video shows what happened and holds no robot actions. It helps pretraining, or feeds a pipeline that adds actions.
What is the difference?
Egocentric data is filmed from a person’s point of view. Robot training data holds one robot’s states and actions, synced to the camera.
How many hours do I need?
Write it as episodes, tasks and variants. Hours alone can hide repetition.
Should I ask for hours or episodes?
Both. Hours compare cost. Episodes are what your training code reads.
Know what you are buying.
Send one spec. We check samples against it before you order.
[+] Sources
- EgoMimic (Kareer et al., 2024). arxiv.org/abs/2410.24221. Human and robot demos co-train with alignment and tracking; tested on three tasks. Read 17 Nov 2026.
- EgoDex (Hoque et al.). arxiv.org/abs/2505.11709 and github.com/apple/ml-egodex. 829 hours, 194 tabletop tasks, Apple Vision Pro capture. Licence CC-BY-NC-ND. Read 17 Nov 2026.
- DROID (Khazatsky et al.). droid-dataset.github.io. 76,000 trajectories, 350 hours, 564 scenes, 86 tasks. Read 17 Nov 2026.
- UMI. umi-gripper.github.io. LeRobot v3 format at huggingface.co/docs/lerobot.