← HomeBuyer’s guide

Egocentric video vs robot training data

Human video shows what happened. Robot data records what the robot did.

Buy human video for pretraining. Buy robot data to train actions on your robot. Drag the line to compare.

A robot gripper holds a mug by its handleA human hand holds the same mug by its handle
Human videoFrames, maybe IMU
Robot dataJoint state and commands, synced to camera

A hand is not a gripper.

A hand, a gripper and a robot arm differ in joints, reach and speed. A head camera sees from eye level, a wrist camera sees the gripper. So human video needs mapping before a robot can learn actions from it.[1]

The scale differs too. EgoDex holds 829 hours of human video across 194 tasks.[2] DROID holds 350 hours of robot data, 76,000 trajectories across 86 tasks.[3]

Six data types, and which hold robot actions.

Check the type before you ask for a quote. Each card opens its suppliers.

Step 4 turns human video into robot data.

If the mapping to your robot fails, the episode stays a video.

01

Record

A person does the task while the camera runs.

OutputTime-stamped video
02

Align

Sensor clocks are synced and the camera calibrated.

OutputSynced streams and calibration
03

Track

Hand, body and head pose are estimated from video.

OutputPose tracks with confidence
04

Map to the robot

Human pose becomes the robot’s joint or gripper space. This needs knowledge of the robot.

If this fails, the episode stays a video
OutputCandidate robot actions
05

Check and label

Start, end, success and gaps are checked.

OutputLabelled episodes
06

Write the format

Saved as LeRobot, RLDS or similar.

OutputDataset files and checksums
07

Train and test

Tested on held-out tasks on the real robot.

OutputA policy and its test results

Put these 11 lines in your spec.

A supplier can only price what you define.

TaskTask, goal, robot, gripper, site.
EpisodeStart, end, success, failure.
SensorsCameras, resolution, fps, sync.
State and actionsWhich joints and commands.
Measured or inferredSay which, field by field.
DiversityTasks, objects, sites, operators.
QualityBlur, occlusion, tracking loss limits.
RightsConsent, use, resale, deletion.
PrivacyFaces, screens, voices removed.
DeliveryFormat, version, loader, checksums.
AcceptanceA paid sample and a pass test.

Test 5 to 10 episodes before you buy.

  • Load them. Use the reader your training code uses. Pass or fail.
  • Check each episode. A start, an end, an action column and rising timestamps.
  • Check sync. Frames and robot state line up, and the calibration file is there.
  • Check fps and resolution. Against your spec.
  • Read the rights pack. Consent wording and a checksum per file.
  • Count failures. Compare them with any rejection rate the supplier states.
  • Pay for the sample. Keep the results with the order.

Know what you are buying.

Send one spec. We check samples against it before you order.

Nothing goes to a supplier without your yes.

A robot arm lifts a test cube above a calibration board while a camera records it
What data do you need?
How much, and by when?
Which robot is it for?
Where should we reply?

Questions

Can I train a robot on egocentric video alone?

For actions on a specific robot, usually not. Video shows what happened and holds no robot actions. It helps pretraining, or feeds a pipeline that adds actions.

What is the difference?

Egocentric data is filmed from a person’s point of view. Robot training data holds one robot’s states and actions, synced to the camera.

How many hours do I need?

Write it as episodes, tasks and variants. Hours alone can hide repetition.

Should I ask for hours or episodes?

Both. Hours compare cost. Episodes are what your training code reads.

Know what you are buying.

Send one spec. We check samples against it before you order.

[+] Sources
  1. EgoMimic (Kareer et al., 2024). arxiv.org/abs/2410.24221. Human and robot demos co-train with alignment and tracking; tested on three tasks. Read 17 Nov 2026.
  2. EgoDex (Hoque et al.). arxiv.org/abs/2505.11709 and github.com/apple/ml-egodex. 829 hours, 194 tabletop tasks, Apple Vision Pro capture. Licence CC-BY-NC-ND. Read 17 Nov 2026.
  3. DROID (Khazatsky et al.). droid-dataset.github.io. 76,000 trajectories, 350 hours, 564 scenes, 86 tasks. Read 17 Nov 2026.
  4. UMI. umi-gripper.github.io. LeRobot v3 format at huggingface.co/docs/lerobot.