Skip to content

Implement Controlled Fault Injection for Localization Replay #763

Open

Description

@ytakano

Implement reproducible injection of input delays, timestamp offsets, dropped messages, and computational overload into localization replay experiments on Awkernel. Exercise the integrated pipeline under controlled abnormal conditions without modifying the original datasets.

Scope:

  • Inject configurable delivery delays and jitter into selected input streams
  • Apply timestamp offsets independently of delivery timing
  • Drop individual messages or configurable bursts using deterministic rules or seeded randomness
  • Introduce controlled computational load through configurable competing tasks
  • Support isolated faults and explicitly defined combinations
  • Record the injection configuration and actual events observed during each run

Acceptance criteria:

  • Each scenario specifies the target stream or task, injection point, start time, duration, and magnitude.
  • Delivery-delay injection preserves measurement timestamps unless timestamp modification is explicitly enabled.
  • Timestamp-offset injection changes only the configured timestamp fields and documents how per-point times are affected.
  • Drop patterns are reproducible, and injected losses are distinguishable from pipeline overflow or processing failures.
  • Computational-load scenarios specify workload, CPU affinity, priority, and activation pattern, with actual interference measured.
  • Replay continues to offer inputs at the configured rate rather than slowing down to match processing completion.
  • Fault-free control runs use the same datasets and baseline configuration.
  • Tests verify that injected events match their specification; deviations caused by scheduling are recorded.
  • Logs retain original and modified timing information, message identifiers, and injection decisions.
  • Scenarios, seeds, configurations, and reproduction commands are documented.

Out of scope:

Defining recovery policies, modifying localization algorithms, physical sensor faults, and vehicle-level safety validation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions