Skip to content

Live camera viewfinder with tap-to-segment interaction - #259

Merged
Monster0506 merged 9 commits into
mainfrom
feat/256-live-camera-viewfinder
Sep 20, 2026
Merged

Monster0506 merged 9 commits into
mainfrom
feat/256-live-camera-viewfinder

Conversation

@Monster0506

Copy link
Copy Markdown
Member

Closes #256

Summary

  • Implemented live camera streaming helper (wwwroot/js/camera-scanner.js) using navigator.mediaDevices.getUserMedia with back-facing camera selection (facingMode: 'environment'), device switching, and error handling.
  • Implemented LiveCameraScanner.razor component with real-time tap-to-segment interaction, freezing frame buffer, and sending tap coordinates to local MobileSAM engine.
  • Implemented responsive SVG animated glowing contour overlay snapping around the segmented object.
  • Added multi-point refinement support (+ Include / - Exclude points) with visual prompt markers.
  • Added action controls: 'Accept item', 'Retap / Cancel', and 'Switch camera'.
  • Added mobile layout protections against pull-to-refresh and accidental zoom gestures (touch-action: none, overscroll-behavior: none).
  • Fixed ONNX image encoder input tensor shape and rank ([H, W, 3] rank 3 HWC raw float RGB values expected by MobileSAM model).
  • Integrated live camera scanner directly into PhysicalAssets.razor review pipeline.
  • Added comprehensive unit tests in tests/js/camera-scanner.test.mjs and tests/MoneyMirror.Tests/PhysicalAssets/LiveCameraScannerTests.cs.

@Monster0506
Monster0506 merged commit 03a1ab1 into main Sep 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Live camera viewfinder with tap-to-segment interaction

1 participant