# Data model

> The five layers Rayform builds from every recording, and the channels and entities that carry them.

Source: https://rayform.world/docs/reference/data-model
Last updated: 2026-10-18
Status: Rayform is in private preview. APIs, formats and the SDK described here may change before general availability.

Every processed segment carries five layers, aligned to one clock and one coordinate frame.

| Layer | What it holds | Where to find it |
| --- | --- | --- |
| Scene | Gaussian splats, meshes and collision geometry of the place | `segment.assets`, `Segment.scene_at()` |
| Motion | Camera, object and body trajectories | Entity trajectories, `poses/*` channels |
| Interaction | Grasps, placements, doors, drawers, tools | `segment.events`, `Segment.events()` |
| Robot state | Joint positions, velocities, effort, gripper | `joints/*`, `gripper/*` channels |
| Context | Lighting, clutter, surface, weather, people | `segment.context` |

## Channel names

Rayform renames source topics to stable channel names, so loaders work the same across robots:

| Channel | Source examples |
| --- | --- |
| `cam{n}/rgb`, `cam{n}/depth` | `/camera/color/image_raw`, `/camera/depth/image_rect_raw` |
| `wrist/rgb` | A camera mounted on the end effector |
| `lidar/points` | `/velodyne_points`, `/ouster/points` |
| `imu` | `/imu/data` |
| `joints/position`, `joints/velocity`, `joints/effort` | `/joint_states` |
| `joints/position_delta` | Derived: change in position to the next sample |
| `gripper/position` | Gripper joint or command topic |
| `poses/{entity}` | Derived trajectories, for example `poses/cam0`, `poses/ent_mug_1` |

The mapping for each session is listed in `session.channels` with the original topic names, so nothing is hidden.

## Entities

| Field | Description |
| --- | --- |
| `id` | Stable within a session, for example `ent_mug_1` |
| `category` | Open-vocabulary label, for example `mug`, `person`, `robot_arm` |
| `moves` | Whether the entity moved during the segment |
| `trajectory` | Poses over time in the site frame, as channel `poses/{id}` |
| `asset` | Splats of the entity alone, when it moves |

## Event types

| Type | Meaning |
| --- | --- |
| `grasp` | A hand or gripper closes on an object |
| `place` | An object is released onto a surface |
| `push` | An object moves under contact without a grasp |
| `open_drawer`, `close_drawer` | Drawer state changes |
| `open_door`, `close_door` | Door state changes |
| `handover` | An object passes between two agents |
| `pour`, `wipe` | Tool actions |

Events carry a span (`t_start_ns` to `t_end_ns`), the instant of contact (`t_ns`), the entities involved, and a confidence.