Bump the exporter to 1.2.1, and build the Windows binary before releasing it - #110
Merged
Conversation
1.2.0 went out at 12:29 UTC on the 16th and the fix for the crash in #89 merged at 14:08, so the current release is the one that cannot get past it - and a second user has now hit it, from a downloaded build rather than a clone. Their error is byte-for-byte identical to the first: "element claims 109 bytes, 61 remain", on a different account and a different operating system. That rules out the reading this had when only one report existed. It is not one unlucky item in sixty-one; a structure of exactly that shape sits in the ProtectedCloudStorage view of unrelated accounts, and every user who has one loses the whole export to it. Nothing new is being shipped - the fix has been on main since the 16th and is verified again here against the pin main now carries, which has moved since. Three changes reach the exporter since 1.2.0: that fix, one FindMy.py pin for the whole repository, and a secret file being usable on Windows. 441 tests, flake8 clean, and `release_version.py --kind exporter --tag exporter-v1.2.1` passes.
… native code A Windows user on 1.2.0 got `OpenTagViewer.exe parou de funcionar` - the process killed by Windows, no traceback, nothing in any log, because a native fault does not raise. Two gaps let that reach them. **Nothing ever built the Windows binary before a release.** The exporter workflow fires on `release: published` and nothing else, so the first run of any .exe was by whoever downloaded it. `exporter-build-check.yml` builds it on pull requests touching python/ and starts it. It uploads nothing and is not part of releasing; its whole job is to fail a PR instead of a download. Windows only for now, because it is the platform that had no pre-release coverage and the one that broke - the steps are otherwise platform-agnostic and the matrix is two lines to widen. **And the self-test proved files were present without running any code.** It checked the pinned roots and anisette's certificate, both of which would have passed here. unicorn loads its native library through ctypes, so `--collect-all unicorn` puts the file in the bundle and nothing establishes that it loads or executes - which is exactly the shape of a hard crash with no Python error to catch. So it now emulates four bytes of ARM64 and reads the register back. Local, instant, no network and no account, and it fails a build rather than a machine. Anisette needs that path to work, since emulating Apple's ADI library is how a sign-in happens without a third-party server. Not a fix: what crashed on that user's machine is still unknown, and the emulator is a hypothesis rather than a diagnosis. This is what turns the next one into a red build.
The first run of this workflow failed with "Failed to spawn: pyinstaller / program not found". `uv sync --frozen` installs the default groups, and PyInstaller is in `build` - so the check built nothing and would have gone green the moment anybody made it non-blocking. Taken from the release workflow's install step verbatim, for the same reason its PyInstaller invocation is: a build check that installs a different set is checking a different bundle.
The first Windows run produced no output at all and exit 127. Every line was still in the buffer when the process was killed: stdout is block-buffered the moment it is a pipe rather than a terminal, which it always is in CI, and a native fault never flushes. That is the difference between "the self-test failed" and "it failed at the emulator", and only the second is worth having. `--version` looked fine for the same reason it always did - it exits normally, so its buffer is flushed on the way out.
The Windows build dies inside the emulator check with no exception - the try/except printed nothing, so there is nothing to catch and nothing in a traceback. The only evidence a hard fault leaves is the last line that got out before it. So each stage announces itself before it runs: import, construct, mem_map, mem_write, emu_start. Whichever has no answer after it is the one that killed the process. The import step also prints where unicorn was loaded from, because a bundle carrying the Python package without the native library beside it is the likeliest cause and looks identical from outside. This is instrumentation, not a fix. It buys one CI round trip and a line number.
…taller The frozen binary gets through import, construct, mem_map and mem_write and dies inside emu_start - so the native library loads and the bundle is not the problem. That leaves two possibilities that look the same from outside: freezing breaks execution, or executing emulated ARM64 does not work on this platform. This runs the same five calls under plain uv Python before anything is frozen. Passing means PyInstaller; failing means local Anisette cannot work on Windows at all, and the sign-in there needs a remote Anisette server - a product decision rather than a build fix. continue-on-error, because the answer is worth having either way and a red X here would bury the frozen result underneath it.
The plain-Python run on the same machine emulates ARM64 and reads back 42, so unicorn works here and the frozen build is what breaks it. What kills the frozen one is still unnamed because bash reports any process Windows killed as 127 - which is bash's own "command not found" and carries no information. PowerShell keeps the NTSTATUS. 0xC0000005 is an access violation and 0xC0000409 is a security check, meaning Control Flow Guard or a stack cookie - and for a library that JITs its own code those point at completely different fixes.
…s not set The previous commit turned this green over a binary that was still dying. A process Windows kills leaves $LASTEXITCODE unset, `exit $null` is `exit 0`, and the step reported success while the self-test never got past emu_start - the log shows the stages and then nothing, no "read back 42" and no "self-test passed". That is the worst outcome available: a check that goes green over the exact failure it was written for reads as coverage and is worse than not having one. Success is now the line the self-test prints when it finishes, and nothing else counts. The exit code is still printed, because it is evidence, but it decides nothing.
Named at last. The frozen binary exits 0xC0000409 from `emu_start` - STATUS_STACK_BUFFER_OVERRUN, which is what __fastfail reports for a security-check failure, and Control Flow Guard is the one in play. Every observation fits: the DLL loads, the emulator constructs, memory maps and writes, and the process dies the moment execution enters code that did not exist at link time and is therefore not in CFG's table of valid indirect-call targets. Two CI runs established it was not packaging. The same five calls under plain Python on the same Windows runner emulate ARM64 and read back 42, so unicorn works there and the bundle carries everything - it is freezing that breaks it. PyInstaller's bootloader is built with CFG on and the flag is process-wide, so no packaging option avoids it. The flag is one bit in the PE header. `scripts/clear_windows_cfg.py` clears it after the build, in both the release workflow and the check, and refuses a file whose headers it does not recognise - because a patch step that silently does nothing produces a binary that looks fixed and still dies. It gives up a real mitigation, and the script says so rather than hiding it. The alternative is an exporter that cannot sign in on Windows at all: emulating Apple's ADI library is how a login happens without handing the exchange to a third-party Anisette server. Eight tests against a synthetic PE, since the real one only exists on a Windows runner - covering the bit, the neighbouring flags it must not touch, both header widths, and the three shapes of wrong file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cuts 1.2.1, which contains the fix for the crash in #89 that 1.2.0 misses by 100 minutes — and adds the CI that would have caught a second, unrelated Windows failure before it shipped.
exporter-v1.2.0publishedSo the current Latest release is the one that cannot get past it.
Why the release is urgent rather than tidy
A second user has now hit the #89 crash, from a downloaded build rather than a clone, and their error is byte-for-byte identical to the first:
element claims 109 bytes, 61 remain, on a different account and a different OS.That kills the reading this had when only one report existed. It is not one unlucky item in sixty-one: a structure of exactly that shape sits in the
ProtectedCloudStorageview of unrelated accounts, and everyone who has one loses their whole export — before a single tag is read, since the failure is insideunlock.Nothing new ships for that. The fix has been on main since the 16th and is re-verified here against the pin main now carries, which has moved since (
102dd8e).And a second, unrelated Windows failure
A Windows user on 1.2.0 reported
OpenTagViewer.exe parou de funcionar— the process killed by Windows, no traceback, nothing in any log, because a native fault does not raise. Two gaps let that reach them:Nothing ever built the Windows binary before a release.
macos-exporter-python.ymlfires onrelease: publishedand nothing else, so the first run of any.exewas by whoever downloaded it.exporter-build-check.ymlnow builds it on PRs touchingpython/and starts it. It uploads nothing and is not part of releasing — its job is to fail a PR instead of a download.And
--self-testproved files were present without running any code. It checked the pinned roots and anisette's certificate; both would have passed here.unicornloads its native library through ctypes, so--collect-all unicornputs the file in the bundle and nothing established that it loads or executes — exactly the shape of a hard crash with no Python error to catch.It now emulates four bytes of ARM64 and reads the register back:
Why both are in one PR
The self-test runs in the release workflow's own smoke step, before any asset is uploaded. Shipping it with 1.2.1 means the 1.2.1 binaries are themselves checked by it — on all four platforms — rather than the check arriving one release too late to cover the release that motivated it.
What is not claimed
The Windows crash is not fixed here, and the emulator check is a hypothesis rather than a diagnosis. A native fault on Windows cannot be reproduced from the machine this was written on. What this changes is that the next one is a red build instead of a photograph.
To diagnose the reported one, the file to ask for is
%TEMP%\OpenTagViewer\exporter.log— the wizard logs there on Windows and flushes per record, so everything up to the fault survives even though the fault itself will not appear.Verified
441 tests pass, flake8 clean,
release_version.py --kind exporter --tag exporter-v1.2.1returns1.2.1. The new workflow's YAML parses and its PyInstaller invocation is byte-identical to the release workflow's Windows step.Relates to #89.
PR description summarised by Claude Code.