Reference

Python SDK

The SDK is a thin layer over the release layout: it resolves versions, verifies checksums and hands you episodes and frames. Everything it does can also be done with pyarrow and a video reader.

Install

The client is distributed from a private index. Index credentials are issued together with dataset access, so the package is only installable by licensees.

Install from the private index

pip install curatrix --index-url https://pkg.curatrix.de/simple

Authenticate

The SDK reads CURATRIX_API_KEY from the environment. Keys are scoped to the datasets, versions and licence tier you hold; a key never widens that scope.

Provide the key

export CURATRIX_API_KEY="ck_live_..."

Open a dataset and iterate

Filter episodes and read frames

from curatrix import Dataset

# Pin the version. Omitting it resolves to the newest licensed release,
# which makes a training run non-reproducible.
ds = Dataset.open("electrical-assembly", version="1.3.0")

print(ds.schema_version, ds.episode_count, ds.accepted_hours)

for episode in ds.episodes(outcome="failure", environment="workshop"):
    frames = episode.frames(stream="head", fps=10)  # lazy, decoded on demand
    recovery = [s for s in episode.subtasks if s.label == "recover"]
    if recovery:
        print(episode.id, len(frames), recovery[0].start_s)

Use the suggested splits

Contributor-disjoint train and eval sets

train = ds.split("train")
evaluation = ds.split("eval")

assert set(train.contributors).isdisjoint(evaluation.contributors)

Without the SDK

Nothing in a release requires our client. The tables are Parquet and the video is MP4.

Read a release with pyarrow only

import pyarrow.parquet as pq

episodes = pq.read_table("episodes/episodes.parquet").to_pandas()
spans = pq.read_table("episodes/annotations.parquet").to_pandas()

failures = episodes[episodes.outcome == "failure"]
print(spans[spans.episode_id.isin(failures.episode_id)]
      .groupby("label").size().sort_values(ascending=False))