Contact Us
Hand UMI

A person holds two handheld grippers, one per hand, and does the task themselves. No robot in the loop — the gripper records what a robot would need to reproduce the motion.

6 Tasks · 8 Variants · 13 Episodes

Pick-and-place, two zips, two lids, two bins and a backpack — three of them one-handed, three needing both grippers at once. Every take below was chosen from a 622-episode delivery on two rigs.

4 Tactile Pads

Each gripper has a camera-based gel pad on both fingers. The gel image is recorded at 120 fps; the vendor derives deformation, shear and depth fields and a six-axis force from it at 60 Hz.

1 kHz Pose · Ego Headset

Both grippers are tracked at about 1 kHz by their own stereo-inertial units and placed in one world frame off an AprilGrid. The operator wears a stereo headset whose head pose runs at 200 Hz.

BEHIND THIS SAMPLE
622episodes · 120 min of task time · 2 rigs

The thirteen episodes below are drawn from a 622-episode delivery recorded over five days on two identical rigs — one on a white table (pens and pencil cases), the other mostly on a black table (boxes, bins and a backpack), with ten bin takes on the white table as well. Every episode passed the vendor's quality check, both gripper trajectories aligned to the AprilGrid, and none is missing a file. Six tasks, eight objects; counts below are exact, every episode was opened.

11%
22%
20%
12%
9%
14%
8%
3%
T168 ep · 39 min
Pens into holder · one hand

White table, 08-07. One to three pens lie on the table; the right gripper drops each into a white holder. The left gripper is carried open. 24–44 s per take.

T2a137 ep · 24 min
Unzip pencil case · yellow

08-11. The left gripper pinches the case body, the right takes the slider and pulls the zip open. 6–19 s per take.

T2b126 ep · 20 min
Unzip pencil case · navy

08-11 / 12. The same two-hand zip on a stiffer navy case with a smaller slider. 7–14 s per take.

T3a73 ep · 6 min
Close box lid · blue box

Black table, second rig, 08-10. A hinged blue box sits open; the left gripper steadies the base while the right flips the lid shut. 4–6 s per take.

T3b59 ep · 5 min
Close box lid · clear box

08-10. The same close on a lighter clear box whose lid snaps — the right pads peak higher than on the blue box. 4–8 s per take.

T490 ep · 14 min
Open backpack zip · two sliders

08-11, second rig. A small backpack with a double-slider zip; each gripper takes one slider and they pull apart in opposite directions. 7–12 s per take.

T648 ep · 9 min
Stand toppled bin up · one hand

08-05 / 10 / 11, second rig. A white bin lies on its side; the right gripper grips the rim, lifts it and sets it upright. Ten takes are on the white table, the rest on the black one. 7–18 s per take.

T521 ep · 2 min
Turn inverted bin upright · one hand

08-10, second rig. A light-blue bin starts upside down; the right gripper grips it, lifts and turns it over. 6–7 s per take.

Task time is the vendor's start/stop window, not the full recording. Both rigs use the same gripper model, the same headset and the same AprilGrid; the second rig's episode numbers overlap the first's, so episodes are identified by UUID. Eight takes in the delivery are exact re-exports of others and are counted once.
PENSone hand · right gripper acts
UNZIPtwo hands · left pins, right pulls
CLOSE LIDtwo hands · left steadies, right flips
BACKPACKtwo hands · sliders pulled apart
BINSone hand · right gripper rights the bin
TACTILE PADS
gel camera on each finger · left gripper first, then right · finger L / finger R are the vendor's names
0.00 / 0.00
STREAMS —
LEFT HAND—
RIGHT HAND—
GRIPPERfinger angle · rad
CONTACT|F| · N
LEFT GRIPPER · FINGER L
LEFT · FINGER R
RIGHT GRIPPER · FINGER L
RIGHT · FINGER R
CLOSED / IN CONTACT
—
Check the synchronisation yourself
Nothing on this page was aligned by hand. Each raw stream carries an epoch start stamp; the page cuts every stream at the same epoch second and lets them run. If the stamps were wrong the pads would load before the fingers close, or the ego view would show a pen already in the holder while the gripper camera still shows it on the table. The table lists what the streams themselves report for each take.
window = [task_start − 1 s, task_stop + 1 s] # vendor stamps, epoch cut(v) = ffmpeg -ss (window[0] − v.start_offset_us/1e6) # per stream closed = angle > min + 0.6·(max − min) # per gripper, own travel contact = hysteresis(max(pad_l, pad_r) |F|, on 1.0 N, off 0.5 N) travel = Σ ‖p[i+1] − p[i]‖₂ # pose_data csv, whole take
The headset keeps its own clock. Before each take the recorder measures the offset between the two and writes the residual to ego_sync.json; across all 622 episodes the residual stays within ±15 ms, under one frame at 60 fps.
The pad force is computed by the vendor from the gel deformation. On the pen task the two right-hand pads agree within a few percent at every grasp — a thin pen loads both fingers alike; on the zip task one finger carries most of the load, which is what a slider pulled against its own tape should do. Neither pattern would hold if the pads were mis-paired.
taketask windowego offsetL travelR travelL peakR peakcontacts L · R
Travel is the path length of the gripper's tracked origin. On the pen task the left gripper still moves about a metre — the operator's idle hand drifts — but its pads never load and its fingers never close; that is the test for "idle", not the travel.