# Preparing recordings

> What to record and how, so Rayform can align, reconstruct and label your data well.

Source: https://rayform.world/docs/guides/preparing-recordings
Last updated: 2026-10-18
Status: Rayform is in private preview. APIs, formats and the SDK described here may change before general availability.

Rayform works with what you already record. Better timing and calibration give better alignment and reconstruction, so a few checks before you record pay off.

## Supported inputs

| Input | Notes |
| --- | --- |
| MCAP (`.mcap`) | Preferred. ROS 2 and Foxglove message schemas are read directly. |
| ROS 2 bags (`.db3`, `.mcap`) | Converted to MCAP on upload. |
| ROS 1 bags (`.bag`) | Supported for common message types. |
| Raw files | Image sequences, `.npy` or `.parquet` tables, plus a `rayform.yaml` manifest describing them. |

## Sensors

Any mix of the following works. More viewpoints and depth give better reconstructions.

| Channel | Message types | Used for |
| --- | --- | --- |
| RGB camera | `sensor_msgs/Image`, `CompressedImage`, `foxglove.CompressedVideo` | Reconstruction, segmentation, training observations |
| Depth camera | `sensor_msgs/Image` (16UC1 mm or 32FC1 m) | Reconstruction, scale |
| LiDAR | `sensor_msgs/PointCloud2` | Geometry, metric scale, registration |
| IMU | `sensor_msgs/Imu` | Pose estimation, time alignment |
| Joint state | `sensor_msgs/JointState` | Robot state and actions |
| Gripper | `control_msgs/GripperCommand` or a `JointState` joint | Robot state |
| Poses | `geometry_msgs/PoseStamped`, `tf2_msgs/TFMessage` | Initial trajectory, extrinsics |

## Timing

Alignment is only as good as your clocks.

- **Best:** hardware synchronisation (PTP or a trigger line) with timestamps taken at capture, not at logging.
- **Good:** one host clock with consistent driver latency. Rayform estimates fixed offsets per sensor from motion.
- **Avoid:** timestamps assigned when messages are written to disk, which drift with load.

Rayform reports the offsets it estimated in the session's `issues`, so you can see how far each stream was moved.

## Calibration

Include intrinsics and extrinsics for every sensor. Rayform reads `sensor_msgs/CameraInfo` and the `/tf_static` tree from the recording. If you calibrate separately, upload the result once per device:

```yaml title="calibration.yaml"
device: arm-cell-1
cameras:
  cam0:
    model: pinhole-radtan
    intrinsics: [612.4, 611.9, 320.1, 241.7]       # fx, fy, cx, cy
    distortion: [0.091, -0.212, 0.0003, -0.0007]
    resolution: [640, 480]
    T_base_cam: [0.12, 0.0, 0.45, 0.5, -0.5, 0.5, -0.5]  # x y z qw qx qy qz
```

```bash
rayform calibration upload calibration.yaml
```

## Raw files manifest

When your data is not in MCAP or a bag, describe it with a manifest next to the files.

```yaml title="rayform.yaml"
site: lab-2
device: arm-cell-1
streams:
  - name: cam0/rgb
    type: image
    path: cam0/*.png
    timestamps: cam0/timestamps.csv     # one int64 nanosecond value per frame
  - name: joints
    type: joint_state
    path: joints.parquet                 # columns: t_ns, position, velocity, effort
    joint_names: [shoulder_pan, shoulder_lift, elbow, wrist_1, wrist_2, wrist_3, gripper]
```

## Capture checklist

- Walk or drive the space slowly at the start, so the static scene is well covered before anything moves.
- Keep at least one camera on the workspace when a robot is acting.
- Record the robot at its normal control rate; do not downsample before upload.
- Note anything unusual (lighting changes, sensor faults) in the session `notes` when you upload.

> Faces and other personal details are blurred before reconstruction, but only record people who have agreed to it, and keep consent records on your side.