Connect the camera

[lbedogni]

Problem
The Raspberry Pi client needs live camera streaming capability for real-time inference. This is essential for any visual recognition use case and provides the foundation for testing and deployment.

Impact

• Without camera support, system cannot perform vision-based inference
• Blocks prototype demonstrations and user testing
• Required for most practical applications

Current State
"The client for the raspberry Pi should connect to the camera and provide a live stream."

Proposed Implementation

Camera Support

Hardware Options

• Official Raspberry Pi Camera Module v2 (8MP, recommended)
• Raspberry Pi Camera Module 3 (12MP, newer)
• USB webcam (plug-and-play)
• CSI camera ribbon (lowest latency)

Camera Configuration

• Configurable resolution (640x480, 1280x720, 1920x1080)
• Adjustable frame rate (15-30 FPS)
• Exposure, white balance, focus settings
• Rotation/flipping for different orientations

Streaming Implementation

Capture Pipeline

• Initialize camera via picamera2 (new) or picamera (legacy)
• Capture frames at configured resolution/FPS
• Minimal preprocessing (optional)
• Feed to inference pipeline

Stream Protocols

• HTTP MJPEG: Browser-viewable live stream
• RTSP: Standard streaming protocol
• Custom protocol: Direct to offloading client

Video Codec Options

• H.264 (hardware accelerated on Pi)
• MJPEG (simpler, less efficient)
• Raw frames (for inference pipeline)

Integration with Inference

Pipeline Flow

1. Capture frame from camera
2. Optional preprocessing (resize, normalize)
3. Run local inference or offload to server
4. Stream results alongside video

Features

• FPS display overlay
• Inference results overlay (bounding boxes, labels)
• Latency display
• Recording capability (optional)

Configuration

Settings

• Camera selection (module vs USB)
• Resolution and frame rate
• Stream protocol and port
• Preprocessing parameters
• Offloading settings

Example Config

camera:
  type: "pi_camera_v2"
  resolution: 1280x720
  fps: 30
  rotation: 0
  
streaming:
  protocol: "http_mjpeg"
  port: 8080
  bitrate: "2000k"
  
pipeline:
  enable_overlay: true
  show_fps: true

Deliverables

• Camera capture module
• MJPEG/RTSP streaming server
• Configuration management
• Web interface for live view
• Performance benchmarks (latency, CPU usage)
• User guide and setup instructions

Dependencies

• Requires Restructure the code #22 (Code restructuring) for modularity
• Enables Tests #17 (Hardware tests), Simulated delays #18 (Simulation tests)
• Prerequisite for ESP32 client #19 (ESP32 client) and Implement mobile application for SCIoT client #23 (Mobile app)

Estimated Effort: Medium (3-4 weeks)

Files: src/clients/raspberry_pi/camera/, src/clients/raspberry_pi/streaming/
Labels: enhancement, "feature", "raspberry-pi"`