Every video frame ships with the exact world state the simulation computed at that instant — every entity, every gameplay component, every world resource. Nothing on this page is inferred from pixels; the numbers come straight out of the engine.
Video, player input, camera pose, full world state, the event log, entity-ID masks, metric depth and 2D boxes all share the 60 Hz simulation tick. Frame N and every data row with tick: N describe the same instant — aligned by construction, not synced afterward.
Every game rule that fires is logged with entity ids and parameters: input fire → rule Shot → rule Damage (−24 HP) → rule Death. Per-moment causal annotation of who did what to whom — supervision that reconstruction-based datasets cannot supply.
Fixed timestep, seeded, per-tick digests: any episode replays bit-identically from its action stream, and any tick can be forked into counterfactual branches with exact ground-truth futures — the answer key action-conditioned world models can't get from real footage.
A game recording that carries its own answer key — per-tick world state, a causal event log, per-pixel labels, and the player’s every input — all read from a deterministic simulation on one clock, never estimated from pixels.
Entity ids are shared across the state stream, the event log, the segmentation masks and the bounding boxes. There is no detection model and no labeling step between a pixel and the truth about the object it shows.
The simulation is a deterministic function of state and input: fixed timestep, seeded, digest-verified per tick. Load the state at any tick, feed a different action stream, and the engine produces the precise ground-truth future of that branch — states, events, and re-rendered video.
Two branches share an identical history and differ by one input flag — the divergence of their futures is pure causation, with an exact answer key. This is the benchmark real footage cannot provide: same past, different action, both futures known.
| Tier | Content |
|---|---|
| Gameplay episodes | Video (MP4/PNG), per-tick actions, camera, full world state, event log. Batch-scale multi-episode runs with consistent schema and per-episode reproducibility metadata. |
| Per-pixel labels | Entity-ID segmentation and metric depth available today; semantic-class segmentation, surface normals and optical flow scoped per engagement. |
| 3D assets | Scenes, characters, vehicles and props as inspectable glTF/GLB, cross-referenced to the entity ids that use them. |
| Custom scenarios | Objectives, agents, hazards, mechanics and environments authored to a specific research question; export schema defined to specification. |
| Simulation access | Counterfactual rollouts and branch-from-any-tick evaluation setups, available on request. |