feat: RAI Semantic Map Memory (rai_semap)

[Juliaj]

Purpose

Supports #225, Design SLAM RAI features

• This PR introduces rai_semap, a semantic map memory system that stores object detections with their 3D locations in the SLAM map frame. The system enables spatial-semantic queries like "where did I see a cup?" and provides persistent memory for robot exploration and task planning.

Proposed Changes

• design doc & readme.
• This is an early-stage implementation. The main goal is to gather feedback on the design and approach before further development. I'm also looking for ideas to deal with following challenges,

  • Deduplication: One of the main challenges is handling multiple detections of the same physical object. The system uses spatial merging with class-specific thresholds, point cloud-based matching when available, and confidence filtering to merge duplicate detections. However, distinguishing between a moved object and a new object instance remains a challenge that needs further refinement.

  • Object Detection Accuracy: The accuracy of object detection directly impacts the quality of the semantic map. The system includes confidence thresholds and bounding box size filtering, but detection accuracy depends on GroundingDINO.

Testing

• New unit tests have been added.
• The current design has been validated with the rosbot-xl demo.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.32%. Comparing base (7626288) to head (9e4e0ee).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #727   +/-   ##
=======================================
  Coverage   65.32%   65.32%           
=======================================
  Files          78       78           
  Lines        3386     3386           
=======================================
  Hits         2212     2212           
  Misses       1174     1174           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.