Skip to content

jaguar2/examples: survive a failed register read under co-channel load - #511

Merged
josephnef merged 4 commits into
OpenIPC:masterfrom
snokvist:jaguar2-read-fail-survive
Oct 30, 2026
Merged

josephnef merged 4 commits into
OpenIPC:masterfrom
snokvist:jaguar2-read-fail-survive

Conversation

@snokvist

Copy link
Copy Markdown
Collaborator

What

An RTL8812BU used as a passive airtime witness (rxdemo, DEVOURER_RX_BUSY_MS=1000, 40 MHz) next to a saturating co-channel 8812EU logged intermittent rtw_read(…) errors, and once aborted with terminate called after throwing an instance of 'std::ios_base::failure'.

Observed:

  • The thread that threw was rxdemo's own DEVOURER_RX_BUSY_MS poller. Its ArmChannelBusy read-modify-writes CLM 0x990/0x994 with no catch around it. The rxdemo thermal and RX-energy pollers had no catch either.
  • The library's Jaguar2 DIG and thermal-track threads already catch. DIG warns on failures 1, 101, 201, …, so most failures appear only as [E] lines.
  • With the libusb rc logged (commit 2), every failure was LIBUSB_ERROR_PIPE in ~100 µs. The device stalls the request: there were no timeouts and no reads slower than 5 ms.
  • It happened on every address sampled: MAC 0x0100, BB config, BB report 0xf08/0xf48, and the RF window 0x2908.
  • An immediate re-read succeeded 6/6. That came from one 90 s run with a throwaway probe build, which is not in this PR.

Not established:

  • Why the chip stalls. The vendor's 0x4E0 write after an always-on-section access (usbctrl_vendorreq, rtw88 rtw_usb_reg_sec) is not ported here, and was not tested as a cause.
  • Whether 20 vs 40 MHz matters independently of airtime (see Measured).
  • That txdemo's thermal sample aborts the same way. That is by inspection, not reproduced.

Change

Four commits, one per defect class:

  1. busy window: a failed arm leaves nothing armed. ClmWindow::arm now disarms before its read-modify-writes. Before, a throw part-way through left an earlier window armed with the new period. busy_window_selftest has two new cases, and the re-arm case fails without the fix. The ArmChannelBusy contract says what each failed access leaves behind, scoped to the Realtek USB backends.
  2. usb: name the libusb error when rtw_read fails. The rc and its libusb name are added to the existing line; the rtw_read( prefix is kept for the scripts that grep it. The measurement is recorded in src/jaguar2/CLAUDE.md with its counterparts. No retry is added.
  3. examples: telemetry pollers survive a failed register read.
    • rxdemo's busy, thermal and energy pollers, and txdemo's thermal sample, now catch, count and skip through examples/common/poll_skip.h, which has the DIG loop's rate-limited warning.
    • A failed busy read still emits rx.busy, with read_failed:1. A failed arm is retried and emits nothing.
    • rx.busy is now in docs/logging.md.
  4. jaguar2: count and rate-limit thermal-track skips like DIG.

If you squash-merge, the commit messages carry the per-class detail. We are happy to split this into separate PRs instead if you prefer: commits 2 and 4 stand alone.

Measured

Rig: one 8812BU (0bda:b812, USB 3) as the witness on ch 149 HT40+. An 8812EU 50 cm away injected MCS1/40/LDPC at saturation with DEVOURER_TX_PWR_OFFSET_QDB=-40 (~1.8k frames/s). Both builds are upstream rxdemo/txdemo; master is d0a9e10. Runs were interleaved master/branch, with the same injector binary throughout.

cell master branch
stressed pollers: 5 ms busy windows, 50 ms thermal, 60 s aborted 2/2, after 173 and 729 windows survived 3/3, ~10.3k windows each, 23 / 23 / 26 failed reads caught
witness cadence: 1 s busy, 1 s thermal, 90 s 0/3 aborted; 2 / 1 / 5 failed reads 0/3 aborted; 6 / 5 / 1 (+2 on the final build)

The fix changes survival, not the failure rate. CLM busy was 73–86% by run.

Counterparts (master witness):

  • With the injector off (ambient, busy ~2%): 0 failures in 45 s.
  • MCS5/40 (busy ~68%): 1 failure in 60 s.
  • MCS1/20 and 6M/20: 0 failures in 60 s each. The 40 MHz witness's CLM read those at only 6% and 29% busy, so 20 vs 40 MHz and airtime are not separated.

Verification: ctest passes, 83/83 with the default build and 86/86 with DEVOURER_MT7612U=ON DEVOURER_PCIE=ON, with no new warnings in touched files.

Limits

  • The data is from one 8812BU, one host, one channel and one interference shape.
  • The 1 s cadence never aborted in 3+3 runs. The stressed cell is what makes the master failure repeatable. A lighter shape passing on master means nothing.
  • A failed register write is still not reported (ctrl_write returns bool and set_bb drops it). An arm can therefore succeed over an engine it did not fully program. That predates this PR; the contract now says so.

Possible follow-ups (not in this PR)

  • Jaguar1 library threads with no catch: PhydmWatchdog, _rxmask_thread, _qd_thread (8814, bare rtw_read32), and _therm_thread. Same defect class, one generation over.
  • Demo paths still unguarded:
    • the main-thread GetRxEnergy in DEVOURER_RX_SWEEP;
    • txdemo's hopset sense window;
    • the station-arm threads (SetStationIdentity readbacks);
    • chanmig / txpower energy and thermal reads.
  • A bounded retry on LIBUSB_ERROR_PIPE in UsbTransport::ctrl_read. It touches every generation, so it needs validating across them.
  • Porting and A/B-testing the vendor 0x4E0 post-access write.
  • Separating channel width from airtime in the failure rate.

Reproduce

Two adapters, each with no kernel driver bound. TX is the injector, WIT the witness:

# injector (any devourer-supported TX adapter; here an 8812EU)
sudo env DEVOURER_PID=0xa81a DEVOURER_CHANNEL=149 DEVOURER_TX_RATE=MCS1/40/LDPC \
  DEVOURER_HOP_BW=40 DEVOURER_TX_GAP_US=0 DEVOURER_TX_PAYLOAD_BYTES=1400 \
  DEVOURER_TX_PWR_OFFSET_QDB=-40 build/txdemo &
# witness: stressed pollers abort master within seconds
sudo env DEVOURER_PID=0xb812 DEVOURER_CHANNEL=149 DEVOURER_BW=40 DEVOURER_CHOFFSET=1 \
  DEVOURER_RX_BUSY_MS=5 DEVOURER_THERMAL_POLL_MS=50 DEVOURER_LOG_LEVEL=warn build/rxdemo

🤖 Generated with Claude Code

snokvist and others added 4 commits October 29, 2026 18:47
ClmWindow::arm set the new period before its register writes and the
armed flag only after them. Each write is a read-modify-write, and a read
that throws part-way over a window still armed (a not-elapsed one the
caller re-arms) left that window armed with the new period, so a caller
that caught the throw and read on could get a reading against a
denominator its hardware never counted. Disarm before the writes.

busy_window_selftest covers both failed accesses: a set_bb whose read
throws mid-arm leaves no reading (fails without the fix: armed 1,
valid 1), and a read that throws changes nothing, so the retry reads the
window. The ArmChannelBusy contract says what each leaves, scoped to the
Realtek USB backends: there a failed read throws, a failed write is not
reported at all, and after a failed arm GetChannelBusy falls back to its
sampled path, so the caller re-arms rather than reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"rtw_read(0a2c), sizeof(T) = 4" could not say whether the device stalled
the request, never answered it, or returned short - the first thing to
know about a failure that comes and goes under load. The line now carries
the rc and its libusb name. (read32_wide's fallback returns a sentinel
rather than throwing and is left as it is.)

What it showed on one 8812BU under a saturating co-channel 40 MHz
injector: LIBUSB_ERROR_PIPE every time, in ~100 µs, on every address
sampled (MAC, BB config, BB report, RF window). An immediate re-read
from a throwaway probe build, not in this tree, succeeded 6/6 in one
90 s run. Recorded with its counterparts in src/jaguar2/CLAUDE.md. No
retry is added: it would change every generation's transport, and one
run does not validate it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rxdemo's DEVOURER_RX_BUSY_MS, _THERMAL_POLL_MS and _RX_ENERGY_MS threads
call the device on their own cadence with no catch, so a register read
that fails under co-channel load is std::terminate. The busy-window one
aborted an 8812BU airtime witness (ArmChannelBusy's read-modify-write of
CLM 0x990). txdemo's thermal sample sits on the TX loop with no catch
around it either - by inspection, not reproduced.

Each now catches, counts and skips the sample through one helper
(examples/common/poll_skip.h): failures 1, 101, 201, ... warned, with the
running count - the library DIG loop's shape. A failed busy read still
emits its rx.busy event, with no reading and read_failed:1, because a
busy-window consumer counts windows; a failed arm emits nothing and is
retried. The thermal and energy series are timestamped samples, so a
skipped one leaves a gap in t (a skipped rx.energy interval drops its
frame counts too - logging.md says so). rx.busy is now in logging.md.

A/B on one 8812BU (USB 3, ch 149 HT40+) next to an 8812EU 50 cm away
injecting MCS1/40 at saturation, -10 dB offset, CLM busy 73-86% by run: with 5 ms
busy windows and a 50 ms thermal poll, master aborted 2/2 within
seconds (173 and 729 windows); this branch ran 60 s 3/3 (~10.3k windows
each, 23-26 failed reads caught across DIG, busy and thermal). At the
witness's own 1 s cadence (thermal 1 s) neither build aborted in 3 runs
each, at 1-5 failed reads per 90 s on master and 1-6 on the branch: the
fix changes survival, not the failure rate. Why the 1 s cadence did not
abort in 3 runs is inferred, not counted: a 1 s poller issues few
transfers next to DIG, so a failure seldom lands on its thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The thermal-track guard warned on every failed tick with no count; the
DIG loop next to it counts and warns on failures 1, 101, 201, ... Same
shape here, so the two read alike in a log and neither floods it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Keep Jaguar2 telemetry running after failed register reads

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• Prevent failed busy-window arms from leaving a stale window available for reading.
• Catch failed telemetry reads so demo pollers continue without emitting misleading samples.
• Identify libusb read errors and rate-limit skip warnings to aid diagnosis.
Diagram

graph TD
  P["Telemetry pollers"] --> U["USB register read"] --> D{"Read succeeded?"} -->|Yes| E["Telemetry event"] --> N["Next poll"]
  P --> W["CLM window"] --> U
  D -->|No| S["Count and warn"] --> N
  W -->|Arm failed| S
  S -->|Busy read marker| E
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Retry failed reads in UsbTransport
  • ➕ Could recover transient stalls before they reach callers.
  • ➖ Changes control-read behavior across device generations.
  • ➖ Does not by itself define what a partially programmed busy window should report.

Recommendation: Keep the PR's poller-level recovery and explicit busy-window state fix: they address process survival and sample correctness without changing every backend's transport behavior. Consider bounded transport retries separately after cross-generation validation.

Files changed (10) +211 / -16

Enhancement (2) +15 / -7
UsbTransport.hInclude libusb diagnostics in read errors +9/-5

Include libusb diagnostics in read errors

• Logs the control transfer's return code and libusb error name, or identifies a short read, before throwing. Preserves the existing rtw_read log prefix.

src/UsbTransport.h

RtlJaguar2Device.cppCount and rate-limit thermal-track skips +6/-2

Count and rate-limit thermal-track skips

• Keeps the thermal-track thread's existing exception recovery while adding a running failure count. Warnings now follow the DIG-style rate limit.

src/jaguar2/RtlJaguar2Device.cpp

Bug fix (4) +107 / -7
poll_skip.hAdd shared telemetry skip counter +39/-0

Add shared telemetry skip counter

• Introduces PollSkips to count failed register accesses and warn on failures 1, 101, 201, and so on. Demo pollers use it to report errors without flooding logs.

examples/common/poll_skip.h

main.cppKeep RX telemetry pollers alive after read failures +55/-6

Keep RX telemetry pollers alive after read failures

• Catches busy-window arm and read failures, thermal failures, and RX energy and quality failures. Failed arms retry without an event; failed busy reads emit a marked rx.busy event, while failed energy intervals drop their frame aggregate.

examples/rx/main.cpp

main.cppGuard TX thermal sampling +8/-1

Guard TX thermal sampling

• Catches and counts failed thermal-status reads in the TX loop. An unsuccessful read produces no thermal sample, and transmission continues.

examples/tx/main.cpp

BusyWindow.hDisarm CLM windows before reprogramming +5/-0

Disarm CLM windows before reprogramming

• Clears the armed state before register read-modify-writes. If a re-arm throws partway through, a previously armed window cannot be read using the new period.

src/BusyWindow.h

Tests (1) +58 / -0
busy_window_selftest.cppTest failed CLM arms and reads +58/-0

Test failed CLM arms and reads

• Extends the register mock to throw during programming or report reads. Verifies that a failed re-arm leaves no reading and that a failed window read remains retryable.

tests/busy_window_selftest.cpp

Documentation (3) +31 / -2
logging.mdDocument failed telemetry interval semantics +2/-1

Document failed telemetry interval semantics

• Documents the rx.busy event and its read_failed marker, including the absence of an event after a failed arm. Clarifies that a failed rx.energy read drops that interval's event and frame counts.

docs/logging.md

IRadio.hSpecify busy-window register failure behavior +11/-1

Specify busy-window register failure behavior

• Documents the Realtek USB behavior of failed arms and window reads, including the sampled fallback when nothing remains armed. Notes that failed register writes are not reported.

src/IRadio.h

CLAUDE.mdRecord co-channel read-failure observations +18/-0

Record co-channel read-failure observations

• Records the measured 8812BU stall errors, polling rates, and comparison runs. Distinguishes those observations from untested explanations and remedies.

src/jaguar2/CLAUDE.md

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Informational

1. Quality and energy failures share one warning 🐞 Bug ◔ Observability
Description
The RX energy thread passes a single PollSkips named DEVOURER_RX_ENERGY_MS to the catch around
GetRxEnergy and the catch around GetRxQuality. When DEVOURER_RXQUALITY is on, both read paths
feed one counter and one rate-limited warning, so the line does not say which read failed.
Code

examples/rx/main.cpp[R2035-2042]

+          devourer::RxQuality q;
+          try {
+            q = dev->GetRxQuality();
+          } catch (const std::exception &ex) {
+            skips.note(*logger, ex);
+            nap(g_rx_energy_ms);
+            continue;
+          }
Evidence
Both catch blocks call skips.note on the same object, so the count and the warning text are shared
between two different read paths.

examples/rx/main.cpp[1919-1935]
examples/rx/main.cpp[2035-2042]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
In the RX energy emitter thread in examples/rx/main.cpp, one PollSkips instance labelled DEVOURER_RX_ENERGY_MS records failures from both GetRxEnergy and GetRxQuality, so the rate-limited warning cannot say which read failed.
## Fix Focus Areas
- examples/rx/main.cpp[1919-1935]
- examples/rx/main.cpp[2035-2042]
## Recommended Fix
Add a second `devourer::PollSkips` with a distinct label (for example "DEVOURER_RXQUALITY") and use it in the GetRxQuality catch block.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can route each severity your way: inline, summary, both, or drop

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread examples/rx/main.cpp Resolved

@josephnef josephnef left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: PR 511 (jaguar2/examples: survive a failed register read under co-channel load)

Verdict: approve once the Qodo thread is closed. The code is sound; the only red check is qodo-gate, which fails on the one unresolved (Optional) thread — the shared PollSkips counter between GetRxEnergy and GetRxQuality. Either give the quality read its own PollSkips("DEVOURER_RXQUALITY") (two lines) or dismiss it in the thread and resolve.

Verified in the tree

  • ClmWindow::arm clears armed_ before the read-modify-writes; read() returns no reading when unarmed and RtlJaguar2Device::GetChannelBusy takes the sampled branch on !_station.armed(), so the IRadio contract text is accurate. The two new selftest cases pass (86/86 on the branch).
  • Every struct the catch paths leave default-constructed (ChannelBusy, ThermalStatus, RxEnergy, RxQuality) has default member initialisers, so the post-catch valid / raw checks in rxdemo and txdemo are well-defined.
  • The energy thread's continue after a failed GetRxQuality skips the adapter.rxpaths emit and lands on the same nap, so the cadence holds.
  • libusb_error_name is in scope in UsbTransport.h; the rtw_read( prefix the scripts grep is kept.
  • CI matrix green; build warnings on the branch are the same pre-existing vendored-header redefinitions as master.

Hardware (this bench)

The stressed cell from the PR body, 60 s each, 5 ms busy windows + 50 ms thermal, ch 149 HT40+, MCS1/40/LDPC at zero gap, -40 qdB:

witness injector build windows failed reads aborted
8812BU 0bda:b812 (USB 2 hub) 8812EU a81a master, PR — — injector died (txdemo aborts at zero gap on this USB 2 hub); busy 0–1%, inconclusive
8812BU 0bda:b812 (USB 2 hub) 8812AU 8812 master 10657 0 no (busy 89%)
8812BU 0bda:b812 (USB 2 hub) 8812AU 8812 PR 10634 0 no (busy 80%)
T3U 2357:012d (8812BU, USB 3) 8812AU 8812 master 5703 0 no (busy 83%)

So the LIBUSB_ERROR_PIPE stall did not occur here in 3 × 60 s at 80–89% CLM busy, on two 8812BU units, with a different injector. The PR's change is still the right shape (a poller must not terminate the process), and nothing regressed: event counts and busy readings match master at 5 ms windows. But the "23–26 failed reads per 60 s" rate is rig-specific — worth saying in src/jaguar2/CLAUDE.md that it was one host + an 8812EU injector, and that another bench (8812AU injector, same load) saw none in 3 min.

Nits

  • src/jaguar2/CLAUDE.md: "from a throwaway probe patch that is not in the tree" is history; keep the measured fact ("an immediate re-read succeeded 6/6") and drop the provenance clause.
  • docs/logging.md rx.busy row is fine; read_failed as the discriminator from an unarmed window is a good call.

@josephnef
josephnef merged commit 34a9e6f into OpenIPC:master Oct 30, 2026
33 of 37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants