Skip to content

Replace quirc and BC-UR libraries with lighter pure-C implementations - #303

Open
odudex wants to merge 6 commits into
Blockstream:masterfrom
odudex:lighter-scan
Open

Replace quirc and BC-UR libraries with lighter pure-C implementations#303
odudex wants to merge 6 commits into
Blockstream:masterfrom
odudex:lighter-scan

Conversation

@odudex

@odudex odudex commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

This PR proposes, for your evaluation, replacing two libraries in the QR scan/display path:

  • components/esp32-quirck_quirc submodule — QR decoding
  • components/esp32_bc-ur (C++) → cUR (C) submodule — BC-UR transport (bytewords, fountain codes, multi-part assembly)

Both were originally developed for Kern and Krux projects. cUR is used envelope-only: all payload CBOR stays in TinyCBOR in main/bcur.c, and main/bcur.h is unchanged, so no callers needed modification. A third small commit makes the libjade ESP_LOG mocks accept non-literal tags (k_quirc logs with a TAG variable).

Measured impact (esp32s3 dev config, vs master)

master this PR Δ
jade.bin 1,554,976 B 1,520,160 B −34.8 KB (−2.2%)
QR decoder flash 30.6 KB 15.2 KB −50%
UR library flash 23.7 KB 12.1 KB −49%
libstdc++ 4.7 KB 1.5 KB −3.2 KB
  • QR decoding is ~2.5× faster on the repo's 13 qr_qvga_* camera fixtures (desktop benchmark feeding the same 220×220 scan input to both decoders at -O2; per-image speedups 1.9–3.2×, decode parity 13/13).
  • QEMU on-device selfcheck: 166.6 s → 144.9 s (−13%). The selfcheck includes non-UR tests common to both builds, so the UR-specific gain is larger than the headline number.

Compatibility and verification

  • The selfcheck bcur tests pass with the original vectors, including the byte-exact comparison of encoded UR:CRYPTO-PSBT parts, wire-format parity with bc-ur. Fountain fragments are additionally cross-validated in cUR's own test suite against foundation-ur-py output.
  • One selfcheck expectation adjusted: cUR dedupes a repeated fragment before counting it, so processed_parts stays at 1 when the same part is presented twice (bc-ur counted 2).
  • Full test_jade.py passes via libjade, including the multi-part UR:CRYPTO-PSBT camera-scan fixture.
  • On-device selfcheck passes under QEMU for both qemu configs (default, and --psram --unamalgamated).
  • Builds verified: esp32 and esp32s3 dev configs, qemu, and the libjade host build.

Caveats

  • Both libraries are young compared to quirc/bc-ur. k_quirc carries a CI-enforced synthetic validation matrix (8k+ decode cases) and a desktop test harness; cUR has a unit-test suite run under ASan/UBSan and validated against foundation-ur-py vectors. Review is very welcome, and I'm happy to address findings in the upstream repos.
  • cUR adds decode-side hardening caps absent from bc-ur (1024 fragments / 256 KiB message, both compile-time overridable).

Motivation

If all goes well, give back from DIY community to the Jade project and at the same time get reviews to help harden these tools.

@jgriffiths

Copy link
Copy Markdown
Collaborator

Hi @odudex awesome stuff!

I have been irked by bc-ur for a long time (we also use it in gdk) - .the c++ impl is inefficient and unwieldy to work with, and the upstream maintainers are resistant to any kind of changes. Its long been a dream of mine to have time to rewrite it as clean C, so its great to see you've done exactly that :)

The code size reduction and performance improvements are also very welcome, so I'm very interested in merging these changes. Swapping from our current impl would require us to fully review/test both libraries, which will take a while, and we'd almost certainly do this a library at a time, starting with bc-ur.

So please bear with us as we review cUR first and then we will look at migrating to it in due course.

@jgriffiths

Copy link
Copy Markdown
Collaborator

Note I've cherry-picked the libjade change into master so you don't have to keep carrying it.

@odudex

odudex commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Great to hear there's a demand for cUR! Your review and feedback are really welcome, and you can count on me to make adjustments if needed.

@jgriffiths

Copy link
Copy Markdown
Collaborator

I'd say for the purposes of keeping this more easily rebase-able, it would be a good idea to structure it as the following:

  • Add cUR component, no other changes
  • Update source to use cUR
  • Add k_quirc component, no other changes
  • Update source to use k_quirc

I'd leave the removal of the existing components out for now since nuking them after the migration is trivial, and rebasing with them deleted will be painful if there are changes there while this is under review.

@odudex

odudex commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Commits structured as requested.

Comment thread main/selfcheck.c Outdated
@odudex

odudex commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Done:

cUR:

  • Refactored the decoder API: the is_complete/is_success/get_last_error trio is replaced by a single state machine - receive_part() now returns the resulting state directly. Terminal states are permanent; all other errors are transient, so scan loops can keep feeding parts past misread QR frames.
  • Replaced double with float where possible (progress estimates are display-only and embedded FPUs are single-precision). Fragment-selection math intentionally stays double: the UR standard requires bit-identical part-index derivation across implementations.
  • Encoder now builds its fountain degree sampler once per message instead of on every part, cutting soft-float ops and allocations from next_part().

Jade:

  • Adapted bcur.c and selfcheck.c to the new API. The state machine collapsed the old success/complete/failure check combinations into single exact-state assertions.
  • Added a second UR commit: the animated-QR progress bar now uses the lib's weighted completion estimate, which also credits not-yet-resolved mixed fountain parts. It has a more linear progress UX.

@jgriffiths

Copy link
Copy Markdown
Collaborator

The state machine collapsed the old success/complete/failure check combinations into single exact-state assertions.

Perfect, this is exactly what I envisioned with the state based API :)

@odudex

odudex commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

cUR had a fix and improvements on recent push.

@jgriffiths

Copy link
Copy Markdown
Collaborator

@odudex I'm dubious about the cUR optimizations added TBH. e.g.:

  • defaulting to adding an 8kb table to make crc calculation slightly faster is nuts for embedded targets (the most likely projects to need a lib like cUR vs the reference impl).
  • In the xor change to use uint64_t - esp32 has no 64 bit registers, so working on integer sizes larger 32bit is always going to be slower since 64 bit is emulated with 2 32-bit registers. Given the following simd changes increase the memory alignment this should probably just be a loop over uint32_t pointers without the memcpy-to-register nonsense (this could use a 64 bit loop for 64 bit cpus).

IMO cUR is too young a project to be complicating the impl with micro-optimizations when you are still making high level changes like design optimizations and API cleanups, and still finding bugs. At the very least these changes should be (1) separate re-implementations of functions, not ifdef'd code complicating the existing functions (2) default to a reasonable size/speed tradeoff and (3) be gated by cpu defines only (so builders don't have to iterate the source files trying to determine which set of platform defines they need to set). I also think you might want to hold micro-optimizations back until the library has been completely tested/reviewed, although that last one is just IMO.

Finally, please note that the github user greenaddress is no longer associated with Blockstream or Blockstream Green in any way.

@odudex

odudex commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

@odudex I'm dubious about the cUR optimizations added TBH

Fair points, most of them taken.

  • CRC32 now defaults to the 64-byte nibble table everywhere. Slice-by-8 is opt-in (still built in CI).
  • The XOR is split into two complete implementations, portable and P4 vector, behind a single top-level ifdef like crc32.c. No more ifdefs inside function bodies.
  • The portable loop chunks by pointer width (uint32_t on 32-bit). The memcpy stays: a raw uint32_t * cast is UB under strict aliasing, and at -O2 the codegen is identical (checked the rv32 disassembly).
  • The P4 path is gated by IDF_TARGET_ESP32P4 with a menuconfig opt-out. The README now has one table with every option, its default, and its cost.
  • Agreed on timing: micro-optimizations are frozen. I'll prioritize fixes and hardening.

For Jade, UR_ENVELOPE_ONLY=ON with default options builds just the transport layer, close to reference complexity.

@odudex

odudex commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

cUR + k_quirc on the right:

Jade_r.mp4

@odudex
odudex marked this pull request as ready for review July 29, 2026 13:44
odudex added 6 commits August 14, 2026 17:49
Add cUR (https://github.com/odudex/cUR), a pure-C implementation of BCR-2020-005 UR encoding, pinned at the head of its main branch. No other changes - the source migration follows in the next commit.
Move the UR transport layer from the C++20 bc-ur library (and its C shim with placement-new sizing constants) to cUR, a pure-C implementation of BCR-2020-005, used as the UR transport envelope only - all payload CBOR remains TinyCBOR in bcur.c.

- main/bcur.c and main/selfcheck.c moved to the cUR API: heap encoder/decoder handles, results borrowed from the decoder, fragments freed with free(). Encoder output is always uppercase.
- collect_any_bcur() now replaces the decoder via qr_data->ctx on hard failure, and bcur_scan_qr() re-reads it after scanning.
- selfcheck: a duplicate part is deduped before being counted, so the 'processed parts' expectation differs when the same part is presented twice.
- Build with UR_ENVELOPE_ONLY to skip cUR's payload-type codecs; libjade links the same component's host static lib (bundled SHA-256, no mbedcrypto/wally dependencies).

The esp32_bc-ur component is left in place (now unused) so it can be removed separately once the migration has settled.
Drive the animated-QR progress bar with ur_decoder_estimated_percent_complete_weighted() instead of the pure received/expected fragment counts. The weighted estimate also credits mixed fountain parts that have not yet resolved a pure fragment, so progress keeps moving where the count-based bar would stall at 'almost done' until the final reconstruction.

Capped at 99% until the decode actually completes.
Add k_quirc (https://github.com/odudex/k_quirc), a rewritten ESP32-optimized QR decoder (bilinear/adaptive thresholding, span-based flood fill, SPIRAM-aware allocation), pinned at the head of its master branch. No other changes - the source migration follows in the next commit.
Rewrite the thin wrapper in main/qrscan.c to the k_quirc_* API, folding quirc_extract + quirc_decode into a single k_quirc_decode() and dropping the caller-allocated datastream scratch buffer (now managed internally). k_quirc_end() is called with find_inverted=false to preserve current behavior. Downstream callers use the qr_data_t / jade_camera_scan_qr / scan_qr wrapper API and are unaffected.

Also update the three build references: main/CMakeLists.txt PRIV_REQUIRES, libjade include dirs, and the libjade.c amalgamated .c includes.

The vendored esp32-quirc component is left in place (now unused) so it can be removed separately once the migration has settled.
- qrscan: test data_type as a bitmask rather than with ==, so codes
  containing a Kanji segment are rejected (PR303-007)
- bcur: make UR_DECODER_ERROR_UNSUPPORTED_SIZE stop the scan, reported
  from bcur_scan_qr rather than the camera callback; move the decoder
  into a stable context struct, removing the refetch-qr_data->ctx
  hazard; show the decoder reset after a checksum failure (PR303-008)
- bcur/qrmode: add bcur_can_create_qr_icons() and check it before
  encoding, escalating QR density or erroring instead of emitting a
  stream the decoder rejects (PR303-008)
- libjade: define K_QUIRC_ADAPTIVE_THRESHOLD and
  K_QUIRC_BILINEAR_THRESHOLD to match the firmware build (PR303-016)
- selfcheck: use strcmp for the bc-ur result type and encoded parts
  1-2; keep part 3 a prefix check but require more than its header
  (PR303-025)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants