Fix dead ill-formed-fourth-byte UTF-8 test sections (byte3/byte4 typo)

[nlohmann]

Summary

Fixes #5416.

The SECTION("ill-formed: wrong fourth byte") blocks in tests/src/unit-unicode3.cpp, tests/src/unit-unicode4.cpp, and tests/src/unit-unicode5.cpp guarded their byte4 sweep with a check on byte3 instead of byte4:

for (int byte4 = 0x00; byte4 <= 0xFF; ++byte4)
{
    // skip correct fourth byte
    if (0x80 <= byte3 && byte3 <= 0xBF)   // <-- checks byte3, should be byte4
    {
        continue;
    }
    ...
}

Because the enclosing loop already restricts byte3 to 0x80..0xBF, this guard was always true, so continue fired unconditionally and check_utf8string()/check_utf8dump() were never actually called — a malformed fourth byte in a 4-byte UTF-8 sequence has never been tested. unit-unicode3.cpp also had a garbled comment ("skip fourth second byte"), fixed here as well.

Fix

Fixed the guard to check byte4 (and fixed the garbled comment in unit-unicode3.cpp), and kept the full byte2 x byte3 combinatorics — matching the style of the sibling "wrong second/third byte" sections in the same files, which also sweep every combination of the bytes they don't directly target. This is a deliberate choice: a smaller, representative-prefix version of this fix was considered (and briefly implemented) on the reasoning that the lexer's next_byte_in_range() validates continuation bytes strictly in sequence with early-exit, so byte4's accept/reject outcome doesn't depend on which specific valid byte2/byte3 values were used to reach it — but the maintainer prefers exhaustive coverage of every byte combination here for a stronger coverage claim, consistent with how the rest of the file already tests this property, so the full sweep is kept.

The companion issue #5418 (further runtime-reduction ideas: the wrong-2nd/3rd-byte sections' combinatorics, the binary-format 4x-parse issue, 16-bit integer sweeps, etc.) is intentionally left out of scope for a separate PR/decision.

Verification (done offline, without CMake/network)

For each file, compiled and ran with:

clang++ -std=c++11 -O1 -w -I include -I tests/thirdparty/doctest -I tests/src tests/src/unit-unicodeN.cpp -o /tmp/out && /tmp/out --no-skip -tce="*downloaded*"

• Proved the section was dead code: on the pre-fix code, running just the "wrong fourth byte" subcase (via doctest's --subcase filter) executes 0 assertions in all three files.

• Proved the fix is meaningful: deliberately reintroducing a bug in the lexer's byte4 range check for the 0xF0 case (temporarily widening it to accept any byte) causes the fixed unit-unicode3.cpp test to fail — confirming the test now actually catches a byte4 validation regression. The bug was reverted immediately after confirming the failure.

• Full-suite assertion counts, before -> after (with the full combinatorial sweep restored):

  File	Before (dead code)	After (full sweep)
  unit-unicode3.cpp	19,501,644	26,579,532
  unit-unicode4.cpp	65,423,652	93,735,204
  unit-unicode5.cpp	14,889,164	17,248,460

  (unicode4's +28.3M matches the issue's own estimate of ~2.36M added iterations for a full-sweep fix.)

• No regressions: all other sections in all three files still pass unchanged, and all three files run clean (0 failed) with --no-skip.

Breaking change?

No. This is a test-only change (no include/ changes) and does not affect the public API in any way.

— opened by Claude Code on behalf of @nlohmann