Skip to content

stereo_mount assembly: SDK-free ZED + Mid-360 recording rig#3070

Draft
jeff-hykin wants to merge 3 commits into
mainfrom
jeff/feat/zed_rec
Draft

stereo_mount assembly: SDK-free ZED + Mid-360 recording rig#3070
jeff-hykin wants to merge 3 commits into
mainfrom
jeff/feat/zed_rec

Conversation

@jeff-hykin

@jeff-hykin jeff-hykin commented Jul 20, 2026

Copy link
Copy Markdown
Member

needed for stereo recording ( DIM-1196 ), blueprints+modules

dimos run stereo-mount-record

… static tf and memory2 recording

- ZedUvcCamera: ZED over plain UVC/OpenCV (no pyzed), splits the side-by-side
  stereo frame into color_image_left / color_image_right at 60 fps
- stereo_mount.urdf: zed pitched up 10 deg in front, mid360 behind pitched
  down 30 deg; datasheet bounding boxes for zed body, zed lenses, mid360
- assembly.py: StereoMountStaticTf republishes the urdf fixed-joint tree on tf
- record.py: StereoMountRecorder (memory2) for both eyes + livox lidar/imu
- blueprints/stereo_mount_record.py registered as stereo-mount-record
- Swap raw Mid360 for PointLio(frame_id=world, sensor_frame_id=lidar_link):
  odometry publishes the moving world->lidar_link edge onto tf
- Rename urdf frame lidar_frame->lidar_link so the odometry edge ties into
  the rig tree (frame names must match exactly for tf lookups)
- StereoMountRecorder now extends PointlioRecorder (odometry-baked poses)

Live-tested: all streams world-anchored (249/254 images with poses), zero
'No direct transform' warnings.
@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
2746 1 2745 29
View the top 1 failed test(s) by shortest run time
dimos.msgs.sensor_msgs.test_image::test_sharpness_barrier
Stack Traces | 0.16s run time
def test_sharpness_barrier() -> None:
        import time
        from unittest.mock import MagicMock
    
        # Create mock images with known sharpness values
        # This avoids loading real data from disk
        mock_images = []
        sharpness_values = [0.3711, 0.3241, 0.3067, 0.2583, 0.3665]  # Just 5 images for 1 window
    
        for i, sharp in enumerate(sharpness_values):
            img = MagicMock()
            img.sharpness = sharp
            img.ts = 1758912038.208 + i * 0.01  # Simulate timestamps
            mock_images.append(img)
    
        # Track what goes into windows and what comes out
        start_wall_time = None
        window_contents = []  # List of (wall_time, image)
        emitted_images = []
    
        def track_input(img):
            """Track all images going into sharpness_barrier with wall-clock time"""
            nonlocal start_wall_time
            wall_time = time.time()
            if start_wall_time is None:
                start_wall_time = wall_time
            relative_time = wall_time - start_wall_time
            window_contents.append((relative_time, img))
            return img
    
        def track_output(img) -> None:
            """Track what sharpness_barrier emits"""
            emitted_images.append(img)
    
        # macOS timer coalescing causes jitter at high frequencies, so use
        # wider windows (2Hz) there. Linux handles 20Hz windows fine.
        _freq = 2 if _IS_MACOS else 20
        from reactivex import from_iterable, interval
    
        source = from_iterable(mock_images).pipe(
            ops.zip(interval(0.01)),  # 100Hz emission rate
            ops.map(lambda x: x[0]),  # Extract just the image
        )
    
        source.pipe(
            ops.do_action(track_input),  # Track inputs
            sharpness_barrier(_freq),
            ops.do_action(track_output),  # Track outputs
        ).run()
    
        time.sleep(0.6 if _IS_MACOS else 0.08)
    
        if _IS_MACOS:
            # At 2Hz all 5 images land in one 500ms window → exactly 1 emission
            assert len(emitted_images) == 1, (
                f"Expected exactly 1 emission (one window), got {len(emitted_images)}"
            )
            assert emitted_images[0].sharpness == 0.3711
        else:
            # Items span 2 windows at 20Hz: items 1-4 in first, item 5 in second
>           assert len(emitted_images) == 2, (
                f"Expected exactly 2 emissions (one per window), got {len(emitted_images)}"
            )
E           AssertionError: Expected exactly 2 emissions (one per window), got 1
E           assert 1 == 2
E            +  where 1 = len([<MagicMock id='139791052068512'>])

MagicMock  = <class 'unittest.mock.MagicMock'>
_freq      = 20
emitted_images = [<MagicMock id='139791052068512'>]
from_iterable = <function from_iterable at 0x7f25e8dcf560>
i          = 4
img        = <MagicMock id='139791051591120'>
interval   = <function interval at 0x7f25e8dcfba0>
mock_images = [<MagicMock id='139791052068512'>, <MagicMock id='139791051954448'>, <MagicMock id='139791051949456'>, <MagicMock id='139791050075808'>, <MagicMock id='139791051591120'>]
sharp      = 0.3665
sharpness_values = [0.3711, 0.3241, 0.3067, 0.2583, 0.3665]
source     = <reactivex.observable.observable.Observable object at 0x7f23a3f923f0>
start_wall_time = 1784587642.1474354
time       = <module 'time' (built-in)>
track_input = <function test_sharpness_barrier.<locals>.track_input at 0x7f23a3ea4040>
track_output = <function test_sharpness_barrier.<locals>.track_output at 0x7f23a3ea40e0>
window_contents = [(0.0, <MagicMock id='139791052068512'>), (0.010113000869750977, <MagicMock id='139791051954448'>), (0.020273447036743...6'>), (0.03042745590209961, <MagicMock id='139791050075808'>), (0.04052853584289551, <MagicMock id='139791051591120'>)]

.../msgs/sensor_msgs/test_image.py:131: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Piggybacks on the pointlio_lidar pose setter as the liveness signal; if no
lidar message lands for 5s the recorder repeats an unmissable error banner
(no lidar -> no odometry -> no world poses) with hardware hints.

Verified: banner fires/repeats with lidar traffic firewalled, silent on a
healthy run (178/182 images world-posed).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant