You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Evaluate Localization Accuracy and Timing Under Faults and Overload #765
Evaluate the integrated Awkernel localization pipeline under controlled input faults and computational overload. Reuse the M2 instrumentation to quantify localization accuracy, end-to-end latency, input data age, deadline misses, and dropped-data rates, including recovery after abnormal conditions end.
Scope:
Establish a fault-free baseline with motion distortion correction enabled
Run documented scenarios involving delivery delays, timestamp offsets, message loss, and computational overload
Evaluate individual faults and selected combinations using the defined input and overload policies
Measure accuracy, latency, freshness, deadline behavior, and data loss throughout each run
Analyze recovery behavior and accuracy–timing trade-offs
Acceptance criteria:
Each experiment records its dataset, ground-truth source where available, configuration, fault schedule, workload, and policy settings.
Position and orientation errors are measured against aligned ground truth; reference-output differences are reported separately.
Timing summaries include median, tail percentiles, maximum observed values, and sample counts.
Input age is reported per relevant stream; timestamp-offset experiments distinguish apparent age from age derived from the preserved original timeline.
Deadline-miss ratios use documented job populations, with unfinished or cancelled jobs accounted for explicitly.
Dropped-data rates specify their denominator and distinguish injected loss, buffer overflow, stale-input rejection, and discarded late results without double counting.
Missing outputs and prediction-only intervals remain visible rather than being excluded from accuracy and latency summaries.
Recovery time and any persistent degradation after fault removal are reported using predefined criteria.
Repeated runs quantify variability, and machine-readable results and reproduction commands are retained.
Out of scope:
New instrumentation or fault-injection frameworks, changes to localization algorithms or scheduling policies, and formal safety or worst-case timing guarantees.
Activity