robonix.primitive.orbbec.camera
primitive package
v0.1.0
Orbbec RGBD camera primitive — multi-model wrapper over the vendored OrbbecSDK_ROS2 driver with config-selected launch file (Dabai DCW, Gemini 330-series), declaring rgb + depth + camera_info topic streams and JPEG camera/snapshot + camera/depth_snapshot MCP tools.
2 issues found while indexing this entry
- capability robonix/primitive/camera/snapshot declares no transport (mcp, grpc or ros2)
- capability robonix/primitive/camera/depth_snapshot declares no transport (mcp, grpc or ros2)
primitive-orbbec-camera-rbnx
Robonix package wrapping Orbbec RGBD cameras — a multi-model driver layer over the vendored OrbbecSDK_ROS2, with the launch file selected by config (launch_file). Validated on Dabai DCW and Gemini 336L (330-series). Owns the primitive/camera/* namespace. Exposes the camera's RGB + depth + camera_info streams under generic contracts plus JPEG snapshot / depth_snapshot MCP tools, so consumers (grasp / detection / scene / vision skills / Pilot / the web client) resolve names through atlas — no hardcoded /camera/color/image_raw paths on the consumer side.
Catalog name: robonix.primitive.orbbec.camera.
Capability surface
| Contract | Mode | Transport | Source / handler |
|---|---|---|---|
robonix/primitive/camera/driver |
rpc | gRPC | Driver(CMD_INIT, config_json) — lifecycle gate |
robonix/primitive/camera/rgb |
topic_out | ROS 2 | /<camera_name>/color/image_raw (sensor_msgs/Image) |
robonix/primitive/camera/depth |
topic_out | ROS 2 | /<camera_name>/depth/image_raw (sensor_msgs/Image; HW-aligned to color when depth_registration=true) |
robonix/primitive/camera/camera_info |
topic_out | ROS 2 | /<camera_name>/color/camera_info (sensor_msgs/CameraInfo) |
robonix/primitive/camera/snapshot |
rpc | MCP | snapshot tool — latest RGB frame as JPEG (sensor_msgs/Image) |
robonix/primitive/camera/depth_snapshot |
rpc | MCP | depth_snapshot tool — depth as 8-bit grey JPEG (preview only) |
camera_info is package-locally defined (see capabilities/primitive/camera/camera_info.v1.toml) because the robonix global tree does not ship that contract yet; codegen + atlas merge package-level capabilities automatically. service-object-detect-rbnx uses it to synchronize RGB frames and, when depth is enabled, to back-project detections. The current vertical-grasp deploy skips depth and lets service-grasp-pose-rbnx map bbox pixels through a calibrated 2D homography.
The snapshot contracts are the on-demand, agent-facing form of the two streams: a persistent subscription keeps the latest raw frame per stream (JPEG encoding happens only when a call arrives), and a call older than one second is served by a one-shot re-read. camera/rgb / camera/depth remain the high-rate data plane for skills; snapshots add a JPEG view for Pilot (the LLM sees the frame as vision input) and the web client. extrinsics stays undeclared until a real handler exists — per the packaging rules, manifest entries must correspond to real declare_ros2_*/@cap.mcp(...) handlers.
Boot ordering
Boot this before any consumer of primitive/camera/*. In the current vertical-grasp pipeline, service-object-detect-rbnx resolves rgb and camera_info during its Driver(CMD_INIT) call. service-grasp-pose-rbnx does not consume camera topics directly; it receives the bbox from detection and uses the deploy's homography.
Driver-init lifecycle
start.sh brings up the atlas bridge — no ROS spawn. The bridge opens a gRPC server, registers the provider, declares only primitive/camera/driver (auto-emitted by the framework when codegen produces a Driver Servicer), then blocks on Driver(CMD_INIT, config_json).
When rbnx boot invokes Init it passes the manifest's config: block as JSON. The lifecycle then runs:
CMD_INIT: parse and validate cfg (camera name, resolution / fps,depth_registration, USB pin, sentinel timeout);CMD_ACTIVATE: spawnros2 launch orbbec_camera <launch_file> …;- wait for the first
sensor_msgs/Imageon the configured RGB topic (sentinel); - declare
primitive/camera/{rgb, depth, camera_info}on atlas; - subscribe both streams so the snapshot RPCs serve the latest frame.
The snapshot MCP tools are declared at startup (before CMD_INIT), the same
way every Robonix provider advertises its capabilities; until the camera is
activated their handlers report that no stream is available.
Atlas only ever advertises endpoints we've confirmed are publishing. CMD_DEACTIVATE drops the snapshot subscriptions and kills the orbbec subprocess; CMD_SHUTDOWN additionally stops the ROS context so the process exits cleanly. Idempotent.
If the camera is not connected or the driver stalls, the sentinel times out and Init returns state="error".
Layout
primitive-orbbec-camera-rbnx/
├── package_manifest.yaml
├── capabilities/
│ └── primitive/camera/camera_info.v1.toml # package-local contract
├── orbbec_camera/
│ ├── __init__.py
│ ├── images.py # sensor_msgs/Image → JPEG (pure, unit-tested)
│ └── main.py # lifecycle + sentinel + snapshot tools
├── tests/
│ └── test_snapshot_images.py # conversion unit tests (no ROS graph)
├── scripts/
│ ├── build.sh # colcon build src + rbnx codegen
│ └── start.sh # source ROS, exec main
└── src/
└── OrbbecSDK_ROS2/ # VENDORED orbbec/OrbbecSDK_ROS2
├── orbbec_camera/ # ROS 2 driver pkg
├── orbbec_camera_msgs/
└── orbbec_description/
Config (passed via Driver(CMD_INIT, config_json))
launch_file: dabai_dcw.launch.py # any launch file the vendored driver ships, per model —
# e.g. gemini_330_series.launch.py (Gemini 335/336)
camera_name: camera # ROS namespace; topic prefix
depth_registration: true # HW-align depth to color frame
enable_d2c_viewer: false # debug RViz; off in deploy
color_width: 640
color_height: 480
color_fps: 10 # dabai_dcw firmware default
depth_width: 640
depth_height: 400
depth_fps: 10
serial_number: "" # pin to specific device when multiple connected
usb_port: ""
sentinel_timeout_s: 30.0 # max wait for first RGB frame in on_activate
snapshot_max_width: 640 # snapshot tools downscale to this width; 0 = sensor resolution
snapshot_jpeg_quality: 85 # JPEG quality, 1..100
snapshot_max_depth_m: 3.0 # depth preview: 0..this range → 8-bit grey (near = dark)
# Topic-name overrides (rarely needed):
# rgb_topic: /<camera_name>/color/image_raw
# depth_topic: /<camera_name>/depth/image_raw
# camera_info_topic: /<camera_name>/color/camera_info
Build / run standalone
bash scripts/build.sh # colcon build vendored src + rbnx codegen
ROBONIX_ATLAS=127.0.0.1:50051 \
bash scripts/start.sh # registers, awaits Init
To drive the lifecycle manually (without rbnx boot), call the camera's Driver service with CMD_INIT and a config JSON, then call CMD_ACTIVATE. Init only validates configuration; Activate returns after the first RGB frame is observed and the three topic streams are declared.
Verification
rbnx caps | grep camera
# Expected: orbbec_camera provider with
# robonix/primitive/camera/{driver, rgb, depth, camera_info, snapshot, depth_snapshot}
ros2 topic hz /camera/color/image_raw # ~10 Hz with default fps
ros2 topic hz /camera/depth/image_raw # ~10 Hz
ros2 topic echo /camera/color/camera_info --once # K, D, R, P intrinsics
Snapshot path (needs Pilot; the tool name is the contract leaf):
rbnx ask '只调用 provider_id=orbbec_camera 的 robonix/primitive/camera/snapshot 获取一帧,并描述你看到了什么;不要执行任何运动。'
The reply should describe the scene, which also proves Pilot attached the
frame as vision input. Without Pilot, any MCP client can call the tool after
resolving it through Atlas/ConnectCapability; the payload is a
sensor_msgs/Image whose data is base64 JPEG.
Vendor / upstream
src/OrbbecSDK_ROS2/ is a verbatim copy of orbbec/OrbbecSDK_ROS2. The Dabai DCW firmware-blob SDK ships under orbbec_camera/SDK/{include,lib}/. If anything diverges from upstream, drop a *.patch alongside src/ documenting the diff.
License
This package: Apache-2.0. Vendored OrbbecSDK_ROS2: see its LICENSE file.