Adventure x#3138
Draft
Nabla7 wants to merge 56 commits into
Draft
Conversation
GalaxeaA1ZAdapter implements the ManipulatorAdapter protocol on top of the vendor a1z SDK (CAN bus, MIT PD + gravity comp at 250 Hz). POSITION (min-jerk planned moves) and SERVO_POSITION (streaming) control modes, latching soft e-stop, FK-based cartesian reads, G1Z gripper support (requires the vendor SDK's 'gripper' branch; meters API, 10 cm measured stroke), and teach-and-play recording exposed as adapter extensions. Startup is a verified zero-force sequence: motors enable at kp=0, every motor must report fresh in-limit feedback before hold gains engage at the measured pose - the vendor's stock start() position-holds a single 50 ms read and snaps the arm to zero if feedback is late (reproduced on hardware). Shutdown re-sends disable frames to all motors including the gripper, whose single vendor disable frame can be lost on a busy bus. Opt out with safe_start=False for vendor-stock behavior. Hardware-validated on a physical A1Z + G1Z.
…sport GsUsbMacBus is a python-can BusABC driving the bundled HHS USB-CANFD adapter over libusb - no SocketCAN, no Linux host. The adapter selects it automatically on macOS (transport="auto"); Linux keeps socketcan. Handles this device's quirks: TX endpoint discovered from descriptors (device uses 0x01, gs_usb lib assumes 0x02), no-op kernel-driver detach, TX echo filtering, RX queue flush on open (the device retains stale frames across sessions, which parse as garbage feedback), and up-to-5s discovery retry (the device drops off the USB bus briefly after a close). Validated on an M4 Pro against a live A1Z: 30 s sustained 250 Hz control loop (p99 cycle 5.2 ms, 0/7500 over the SDK's 12.5 ms watchdog limit), ~100% feedback from all 7 motors. Requires pyusb + gs_usb + libusb.
coordinator-galaxea-a1z runs the arm under the ControlCoordinator with a trajectory task. Hardware-certified end to end: coordinator boot, client-submitted multi-joint trajectories over LCM RPC, 0.07 rad return drift. Defaults to the stable arm-only configuration (SDK main branch). The vendor gripper branch ships a G1Z gravity model that mismatches at least some mountings - it pushes the arm during the zero-force startup window, and disabling gravity compensation as a workaround leaves the vendor's soft e-stop unable to catch the arm. Flip gripper=True after gravity/ zero-point calibration (vendor tools/set_zero.py).
…dware Wire the galaxea_a1z hardware adapter into the a1z planning model that landed on main (dimos/robot/manipulators/a1z): a planner+coordinator blueprint for ManipulationModule-driven motion and a keyboard teleop blueprint using the eef twist task. Both run the arm-only stable configuration (gripper=False, A1Z_Flange model) consistent with coordinator-galaxea-a1z; the mock-based a1z-* blueprints are untouched.
…available The A1Z ships with an HHS USB-CANFD adapter that the kernel gs_usb driver cannot actually drive: the adapter's VID/PID (a8fa:8598) is not in the driver's device table, and its bulk OUT endpoint is 0x01 where kernels before 6.x hardcode 0x02, so every transmit fails even if the driver is force-bound. On such machines (found on a JetPack 6 Jetson, kernel 5.15) the arm is unreachable over socketcan no matter what, even though the adapter is plugged in and healthy. Teach transport="auto" to handle this: keep socketcan whenever the channel exists and is up, so stock kernels that bind the adapter keep working exactly as before, and only otherwise fall back to the userspace gs_usb bus already used on macOS, with a log line saying so. A channel that merely exists is not enough to pick socketcan: an unrelated on-board CAN controller can expose can0 with nothing wired to it, which is exactly what the Jetson does. Verified on hardware: zero-force scan reads all 7 motors, the coordinator blueprint boots and holds pose, and the first-trajectory script passes with 0.066 rad settled drift.
Keep the gripper powered during hand-teaching and toggle it open/closed with a single g keypress, so the operator's hand stays out of the wrist camera footage. The legacy pinch-by-hand behavior remains available via --gripper-free-drive. Prompts now read single keypresses (cbreak) with a line-based fallback for non-tty stdin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The HHS adapter is a Full-Speed USB device read one frame per libusb round-trip. Draining it synchronously from the SDK's 250 Hz control loop (~1 ms budget per cycle) cannot sustain the ~3500 frames/s the bus carries (7 motors' feedback plus TX echoes), so the device FIFO overflowed and feedback froze for 0.3-2.4 s at a time - poisoning teach recordings with stale-then-jump positions and tripping the SDK's stale-feedback watchdog during replay. A dedicated reader thread now drains USB continuously (libusb releases the GIL while blocked), filters TX echoes, and queues real frames; recv() becomes an instant pop that always meets the SDK's drain budget. Validated on hardware: teach recordings are freeze-free and replay runs without stale warnings.
Two changes so a hand-taught demonstration replays at 1.0x instead of being silently time-scaled to a crawl: - Smooth on the uniform resampled grid instead of the raw samples. The recorder's sample spacing is irregular (10-100 ms under load); differentiating linear interpolation across those gaps manufactured acceleration spikes an order of magnitude above the real motion (74 rad/s^2 measured vs ~14 real), which throttled the safety time-scaler and put velocity ripple into the streamed commands. Two moving-average passes give a continuous velocity profile. - Size the replay caps to measured reality: natural hand teaching peaks ~3.5 rad/s (old cap 1.5 rescaled every demo), staying under the SDK's 4.0 rad/s streaming cap and 7-20 rad/s watchdogs. The gripper is commanded through the SDK's own controller (measured ~0.28 m/s, ~9 m/s^2), so its caps now cover the SDK's own motion. Validated on hardware: a 33.5 s natural-speed episode preflights at 1.00x and replays smoothly at 0.5x and 1.0x.
Single-purpose keys with a persistent status line: SPACE starts and saves episodes, g toggles the gripper, d discards, q quits with a save/discard confirmation when an episode is in progress. Denoise the session by fixing root causes: publish the camera transform immediately at start (first-second frames also gain poses), rate-limit the recorder's poseless-message warning, drop in-flight recording callbacks during shutdown to avoid the closed-database error, and treat disabling an already-disconnected A1Z adapter as success. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add local_only (default true) so multicast scouting is pinned to the loopback interface and listen endpoints stay on localhost. Prevents two machines on the same LAN from discovering each other and fighting over shared topic names.
This reverts commit a951ec0.
Review feedback on the manipulation docs: a new developer cannot tell what DimOS offers for manipulation or how to build an application with it. The index was a feature braindump leading with keyboard teleop; the learning stack (the meat) was absent from Mintlify entirely; per-arm integration write-ups sat in the capability section. Restructured around what a developer wants to do: - index.md — rewritten. What the stack is, a choose-your-workflow table, the three layers, and why the planning layer is where your time goes. Two-minute quickstart against the mock arm. - planning.md (new) — the planning stack: how a pose target becomes motion, the plan/preview/execute API, backend selection, and how to plug in your own planner or IK solver (protocol + factory, no registry). Ends with where learned policies fit: a policy replaces the planner, feeding a servo task instead of a trajectory task. - learning.md (new) — the full loop on one page: demonstrate (Quest VR collect or A1Z hand-teach) → session.db → dataprep (LeRobot v3/HDF5) → lerobot-train → run the policy on the robot. Every command verified against this branch, including the new a1z teach/replay/run-policy CLI and undo-last-episode. - a750.md and openarm_integration.md moved to platforms/arms/ — they are platform setup docs, not capability docs. Redirects added. VR teleop and data collection are now first-class in the manipulation docs (learning.md) instead of hidden in dimos/teleop/quest/README.md.
Zenoh peers discover each other via multicast scouting, which is bidirectional and (with a tailscale 224.0.0/4 route) leaks across the tailnet, so separate machines collide on shared topics. Hash a per-machine domain (default: hostname) to a multicast port so each machine lands on its own discovery port. Same-machine processes share the port and still auto-discover; different machines never see each other's traffic. Set DIMOS_ZENOH_DOMAIN to the same value on machines that should share (like ROS_DOMAIN_ID). Multicast stays on; no coordinator changes.
This reverts commit 6f27e91.
Peers only form a transport link when they share the secret, so other machines on the network can't see or collide with this host's zenoh traffic. The password defaults to the machine id (derivable by any on-machine process, so local peers auto-share) and can be overridden with DIMOS_ZENOH_PASSWORD.
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| def _password_dictionary_file(user: str, password: str) -> str: | ||
| directory = CACHE_DIR / "zenoh_auth" | ||
| directory.mkdir(parents=True, exist_ok=True) | ||
| digest = hashlib.sha256(f"{user}:{password}".encode()).hexdigest()[:16] |
| directory.mkdir(parents=True, exist_ok=True) | ||
| digest = hashlib.sha256(f"{user}:{password}".encode()).hexdigest()[:16] | ||
| path = directory / f"{digest}.txt" | ||
| path.write_text(f"{user}:{password}\n") |
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
❌ 4 Tests Failed:
View the top 3 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
Collaborator
Author
|
@jeff-hykin can you explain a bit here the changes to core/zenoh that were made? |
…ure_x # Conflicts: # dimos/robot/cli/dimos.py # docs/docs.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contribution path
Problem
Solution
How to Test
AI assistance
Checklist