robonix.skill.jaka

skill v0.1.0

JAKA Mini grab/release + camera-driven pick skill.

View source Open README on GitHub JSON
README skill-jaka-rbnx@main

skill-jaka-rbnx — JAKA Mini grab/release/pick skill

Robonix skill for the fixed-base JAKA Mini + TG-9801 gripper. Three synchronous MCP tools:

  • jaka_grab — open gripper → hover → descend to the configured grasp pose → close gripper → settle → lift back to hover and hold. No perception; poses come from the deploy config.
  • jaka_release — open the gripper in place, zero arm motion.
  • jaka_pick — locate a named object with the camera (yolo11-obb or open-vocabulary VLM backend, detect_scheme) and grasp it: 2D hand-eye homography maps the detection to arm base_link XY. Requires on-site calibration (homography_matrix, detector config — see config.spec); reports a clear config error until then.

Everything is driven through the jaka_arm primitive's ROS 2 contracts (pos_command / joint_command), resolved via atlas. See CAPABILITY.md for frames, units and known limitations.

Build

rbnx build -p ../skill-jaka-rbnx        # from the deploy repo
# or standalone (needs grpcio-tools on the codegen interpreter):
RBNX_CODEGEN_PYTHON=/home/nvidia/robot-jaka-mini/.venv/bin/python \
  rbnx build -p /home/nvidia/skill-jaka-rbnx

Outputs rbnx-build/codegen/{proto_gen, robonix_mcp_types, ros2_idl}.

Boot

The deploy manifest declares the skill in its skill: section; soma stage 2 sends only CMD_INIT (skill stays INACTIVE) and the executor activates it on the first tool call. Static checks after bash start.sh:

rbnx caps -v                              # jaka [INACTIVE] until first call
rbnx tools                                # jaka_grab / jaka_release / jaka_pick listed
rbnx contracts -p robonix/skill/jaka
rbnx describe --provider jaka

Staged acceptance

  1. Read the current TCP pose for calibration: ros2 topic echo /jaka/end_pose --once (ROS_DOMAIN_ID=0 RMW_IMPLEMENTATION=rmw_zenoh_cpp).
  2. release first (gripper moves, arm must not): rbnx ask "open the gripper" — sample end_pose before/after; position delta must be ≤ 2 mm. Observe the fingers opening.
  3. grab with small descent steps: set grasp_pose x/y/rpy from step 1, hover z = 0.275, then lower grasp z stepwise 0.20 → 0.175 → 0.15 → 0.125 (AIIT verified desktop height, config.yaml:34), rebooting between edits.
  4. Write the calibrated hover_pose / grasp_pose back into the deploy manifest.

Safety notes

  • The skill assumes jaka_arm.config.motion_enabled: true. With it false every command — including the gripper — is silently rejected; pose waypoints time out with an explicit hint, gripper commands do not (no feedback path exists).
  • Pose-timeout error text always names motion_enabled as the first suspect when the arm never moved.