tutorial.title

tutorial.description

Step 1 — Evaluate a sample

Before accessing a full release, evaluate the manifest of a sample dataset. The manifest tells you the schema version, the episode count, the accepted hours, and the file map. Reading it first saves you from discovering a format mismatch after downloading a large release.

Fetch and inspect the manifest

# 1. Fetch the sample manifest (replace with your download URL)
curl -H "Authorization: Bearer $CURATRIX_API_KEY" \
  https://api.curatrix.de/v1/datasets/electrical-assembly/versions/1.3.0/manifest \
  -o manifest.json

# 2. Inspect the top-level keys
python - <<'EOF'
import json
m = json.load(open("manifest.json"))
print("dataset:", m["dataset_id"], "version:", m["version"])
print("episodes:", m["episode_count"], "accepted_hours:", m["accepted_hours"])
print("schema_version:", m["schema_version"])
EOF

Step 2 — Load episodes and annotations

Episodes and annotations are standard Parquet. No SDK required — any Parquet reader works. Start with the episode table to understand the outcome distribution and environment breakdown before touching the annotation table.

Load episodes (pyarrow)

import pyarrow.parquet as pq
import pandas as pd

episodes = pq.read_table("episodes/episodes.parquet").to_pandas()
print(episodes.dtypes)
print(episodes.head())

# Outcome distribution
print(episodes["outcome"].value_counts())

# Filter to success episodes in workshop
success_workshop = episodes[
    (episodes["outcome"] == "success") &
    (episodes["environment"] == "workshop")
]
print(f"{len(success_workshop)} success/workshop episodes")

Then load the annotations table and join on episode_id:

Step 2b — Annotation detail

The annotations table has one row per subtask span. Spans are contiguous within an episode and cover it completely. The label column uses the published vocabulary; span_index gives the ordering within the episode.

Subtask duration and recovery detection

import pyarrow.parquet as pq

spans = pq.read_table("episodes/annotations.parquet").to_pandas()

# Subtask duration analysis
spans["duration_s"] = spans["end_s"] - spans["start_s"]
print(spans.groupby("label")["duration_s"].describe().round(2))

# Find episodes with recovery sequences
recovery_ids = spans[spans["label"] == "recover"]["episode_id"].unique()
print(f"{len(recovery_ids)} episodes contain a recovery sequence")

Step 3 — Use the Python SDK (optional)

The SDK is a thin layer over the Parquet tables. It handles version resolution, checksum verification, and lazy frame decoding. If your stack already reads Parquet and MP4 directly, you do not need it.

Filter episodes and read frames with the SDK

from curatrix import Dataset

# Pin the version — omitting it resolves to the newest licensed release,
# making a training run non-reproducible.
ds = Dataset.open("electrical-assembly", version="1.3.0")

print(ds.schema_version, ds.episode_count, ds.accepted_hours)

for episode in ds.episodes(outcome="failure", environment="workshop"):
    # frames() is lazy — decoded on demand, not all loaded at once
    frames = episode.frames(stream="head", fps=10)
    recovery = [s for s in episode.subtasks if s.label == "recover"]
    if recovery:
        print(episode.id, len(frames), recovery[0].start_s)

Step 4 — Run an evaluation and benchmark

The benchmark target should have been agreed before collection started. Lock the evaluation split before training — use the contributor-disjoint split shipped in the release. Record the dataset version, the schema version, the split hash, and the benchmark result alongside your training run so the result is reproducible.

Evaluate on the contributor-disjoint split

from curatrix import Dataset
import numpy as np

ds = Dataset.open("electrical-assembly", version="1.3.0")

# Use the supplied splits — contributor-disjoint by construction
train = ds.split("train")
evaluation = ds.split("eval")

assert set(train.contributors).isdisjoint(evaluation.contributors), (
    "Splits share contributors — evaluation would measure memorised scenes."
)

print(f"Train:  {len(list(train.episodes()))} episodes")
print(f"  Eval: {len(list(evaluation.episodes()))} episodes")

# ── Run your model on the eval split ──────────────────────────────────────
# Your benchmark target was agreed before collection started.
# Record: dataset version, schema version, eval split hash, benchmark result.

benchmark_result = your_model.evaluate(evaluation)  # replace with real call
print(f"Benchmark: {benchmark_result:.4f}")

# Compare against the baseline you locked before collection.
# If the gap is meaningful, use it to scope the next targeted collection round.

Where evaluation exposes a gap, use it to scope the next targeted collection round. A gap in recovery sequences, for example, translates directly into a request for more failure-and-recovery episodes in a specific subtask.

tutorial.docsCallout.title

tutorial.docsCallout.description

tutorial.docsCallout.cta tutorial.docsCallout.ctaContact

Start with a scoped pilot

Bring a task. On the call we scope the capture protocol, the episode volume, the annotation schema and the delivery format.

Book technical call Request custom dataset

Pilots typically start from approximately €25,000. This is an indicative figure only—final scope and pricing are set after a technical qualification call.