robot-jaka-mini_with_tg9801_gripper — JAKA Mini Robonix deployment

Robonix deployment for the fixed-base JAKA Mini 6-DOF arm with a TG-9801 adaptive gripper (end RS485). The arm is driven by the primitive-jaka-mini_with_tg9801_gripper-arm_gripper-rbnx package over TCP to the controller (jkrc SDK).

Hardware & platform

Packages

The committed manifest references all packages by url: — catalog-resolvable for the package site; rbnx boot fetches them into rbnx-boot/cache/:

On the deploy machine, a gitignored robonix_manifest.local.yaml (same content, path: instead of url: for the three packages above) makes build.sh / start.sh build and boot the sibling checkouts (../primitive-*-rbnx, ../skill-jaka-grasp-rbnx) directly — no GitHub fetch, unpushed local edits take effect. robot_description has no local checkout and stays url: (built from rbnx-boot/cache/; refresh with rbnx update).

Pick skill (camera-driven grasp)

jaka_pick locates a named object and grasps it: resolve camera/rgb → grab a frame → detect → a calibrated 2D hand-eye homography maps the bbox centre to arm base_link XY → grasp at (x, y, desktop_height) with a vertical-down orientation. It reuses the same grasp segment as grab.

Two detection backends (detect_scheme in the skill config):

• yolo (deploy default): in-process yolo11-obb — the fruit model (carrot/potato/tomato) reused from the agilex deploy at robot-agilex-robonix/host-services/yolo/weights/best.pt (point the YOLO_MODEL_PATH env var at it — the manifest forwards it to yolo_model_path; the skill interpreter has CUDA torch + ultralytics). Two-frame consistency filters flicker; Chinese phrasing maps to class names via yolo_class_aliases (胡萝卜→carrot …); the OBB rotation aligns the gripper yaw to the object's long edge (rz = grasp_yaw_offset_deg - yaw, default -90 = this arm's tuned value — adjust by ±90 if the fingers land off-axis).
• vlm: open-vocabulary multimodal detection (pixel bbox, no rotation → fixed grasp_rpy).

Calibration is required and on-site. pick is unavailable until you set, in robonix_manifest.yaml's skill: - name: jaka config block: - homography_matrix — 3×3 pixel→base_link, calibrated at 1280×720 (same kind of matrix as agilex's grasp_pose); - the scheme's detector: the YOLO_MODEL_PATH env var (weights file, exported like the VLM vars — e.g. in .vlmkey) or vlm_base_url / vlm_api_key / vlm_model (multimodal endpoint); - desktop_height — the table height in base_link (grasp z).

Until then pick returns a clear error; grab / release are unaffected.

Calibration

jaka_pick's homography_matrix is measured on site with the in-tree bench under tools/calibration/ — a cv2-GUI tool (served as a browser page on headless machines) for this JAKA Mini + Gemini 336L pair. The arm side reuses primitive-jaka-mini_with_tg9801_gripper-arm_gripper-rbnx's JakaDriver (probed from the sibling checkout or rbnx-boot/cache/), so the tool ships no jkrc/gripper code and no SDK binaries; the intrinsics (camera_matrix / dist_coeffs) ship as data in tools/calibration/intrinsics-data/.

Run it with robonix stopped (the camera and the controller are single-owner), workspace clear, estop at hand:

bash tools/calibration/setup_venv.sh
cd tools/calibration && .venv/bin/python calibrate_handeye_2d.py            # default: calibrate
cd tools/calibration && .venv/bin/python calibrate_handeye_2d.py --mode test

The tool prints the fitted matrix as ready-to-paste YAML rows; hand-copy it into homography_matrix in both robonix_manifest.yaml and robonix_manifest.local.yaml. desktop_height is not produced by the tool. The matrix is only valid at 1280×720 on undistorted frames — the tool pins the stream and warns loudly on any fallback. Full flow and gotchas: tools/calibration/README.md (Chinese).

Estop recovery

After the emergency stop is released (pendant knob pulled up; the controller is usually powered off), tools/jaka_estop_recover.py resets it over raw jkrc: clear error → power on → enable → confirmation → slow move to Home (all-zeros) → disable and power off. It reuses the calibration bench's plumbing (arm package probe, arm_port from tools/calibration/config.yaml, the Python 3.10 venv) and bypasses the runtime's motion_enabled gate — run it with robonix stopped:

tools/calibration/.venv/bin/python tools/jaka_estop_recover.py

URDF

urdf/robot.urdf is the JAKA app export of the real JAKA Mini (with meshes/). It is consumed by Soma (urdf.path in soma.yaml), which serves the raw XML to robot_description via get_urdf.

The export was post-processed for this deployment: the 6 arm joints were renamed link_002_joint..link_007_joint → joint_1..joint_6 so robot_state_publisher matches the joint_states names published by the jaka_arm provider. If the URDF is re-exported from the JAKA app, re-apply this rename (see the note in soma.yaml). The two gripper finger joints (link_008_joint/link_009_joint) are kept as exported; the provider publishes a single gripper_joint value.

Prerequisites

• Robonix tooling: make install from a cloned syswonder/robonix checkout, then rbnx setup <robonix source dir>.
• sudo apt install ros-humble-rclpy ros-humble-rmw-zenoh-cpp ros-humble-nav-msgs ros-humble-sensor-msgs (nav-msgs/sensor-msgs feed the Soma runtime reader — see the runtime_reader_command comment in the manifest)
• Primitive package built: bash build.sh (builds the sibling checkouts via the local manifest; a first build on a fresh clone fetches the url-based packages into rbnx-boot/cache/)

Build

bash build.sh

build.sh is the canonical entry: it prepares the deploy venv, then builds every package declared in the manifest (the local path: variant when present, else the committed url: one) with --no-update-check so builds stay offline. robot_description is fetched from the package catalog into rbnx-boot/cache/ when missing.

Camera build note (native): the Orbbec driver (../primitive-orbbec-camera-rbnx) colcon-builds the vendored OrbbecSDK_ROS2, which needs ROS 2 C++ deps (rclcpp, image_transport, cv_bridge, camera_info_manager, tf2_ros, …) absent from this Jetson's minimal /opt/ros/humble. Install them once per machine (needs sudo):

bash ../primitive-orbbec-camera-rbnx/scripts/install_deps.sh

The camera build.sh checks for these and fails fast with this pointer if they're missing. The arm-only stack builds fine without them (the camera primitive is only needed for pick).

Boot

# standard entry — committed url manifest, packages from rbnx-boot/cache/
rbnx boot

# offline / pinned cache: skip the pre-boot per-package `git fetch`
rbnx boot --no-update-check

# after estop verification and single-joint tests, enable motion in
# robonix_manifest.yaml: jaka_arm.config.motion_enabled: true

Motion commands stay rejected while motion_enabled: false (safety gate, see ../primitive-jaka-mini_with_tg9801_gripper-arm_gripper-rbnx/config.spec).

Bare rbnx boot resolves every package to its rbnx-boot/cache/ clone (no sibling checkouts needed); the jkrc SDK ships inside the arm package (Apache-2.0), so no manual copy is needed. Per-machine prerequisites:

• .vlmkey vars in the booting shell — bash start.sh sources them, bare boot cannot (this machine's ~/.zshrc does).
• on a fresh machine, bash build.sh once before the first boot — it prepares the deploy venv and the codegen interpreter the packages' builds need (rbnx boot's inline-build fallback does not set it).

The pre-boot freshness check only reports; it never pulls. When it marks a provider behind, run rbnx update (needs network) before booting.

The manifest's env.PATH expands ${PWD}/.venv/bin — the venv built by build.sh that gives the robot_description provider a new enough grpcio for its generated bindings; boot from this directory.

Development against unpushed sibling edits uses the gitignored robonix_manifest.local.yaml (path: entries): build.sh and start.sh prefer it (start.sh also sources ROS + .vlmkey and owns the zenoh-router lifecycle). Push, then rbnx update, to give bare boot the same code.

Acceptance record

Fill in the robot revision, Robonix commit (git -C <robonix> rev-parse HEAD) and measured gripper open reading here after acceptance.