Annotation schema

Episode record

{
  "episode_id": "ep_000123",
  "schema_version": "2.1",
  "task": "terminal-block-wiring",
  "environment": "workshop",
  "outcome": "failure",
  "subtasks": [
    {"label": "approach", "start_s": 0.00,  "end_s": 3.40},
    {"label": "grasp",    "start_s": 3.40,  "end_s": 5.10,  "object": "obj_2"},
    {"label": "align",    "start_s": 5.10,  "end_s": 9.80,  "object": "obj_1"},
    {"label": "actuate",  "start_s": 9.80,  "end_s": 21.20, "tool": "t_1"},
    {"label": "verify",   "start_s": 21.20, "end_s": 25.00, "result": "reject"},
    {"label": "recover",  "start_s": 25.00, "end_s": 38.70},
    {"label": "release",  "start_s": 38.70, "end_s": 40.10}
  ],
  "tools": [
    {"id": "t_1", "type": "screwdriver", "spec": "ph1-vde"}
  ],
  "objects": [
    {"id": "obj_1", "type": "terminal-block", "spec": "din-rail-2.5mm2"},
    {"id": "obj_2", "type": "wire", "spec": "h07v-k-1.5mm2"}
  ]
}

Subtask vocabulary

Spans are contiguous and cover the whole episode. Labels come from a closed vocabulary: a dataset may use a subset, but never a label outside it.

• transit — moving between locations without holding the target object.
• approach — moving the hand toward the object or workpiece.
• grasp — establishing a grip.
• align — positioning the object or tool before actuation.
• actuate — the force-applying part of the task.
• verify — checking the result; carries a result field.
• recover — corrective action after a failed verify.
• release — letting go and withdrawing.

Outcome states

Failure and recovery episodes are kept deliberately. They are the hardest demonstrations to collect and usually the most useful to learn from, so they are labelled rather than filtered out.

• success — the task completed and the verify span passed.
• failure — the attempt never reached a passing verify.
• recovery — a verify failed and a recover span brought the attempt to a passing state.

Schema versioning

schema_version is major.minor. A minor bump adds optional fields or vocabulary entries and stays readable by older loaders. A major bump can change or remove a field, and always comes with a new major dataset version.