[DO NOT MERGE] Validate the signing config for #23261

[mokagio]

Validation only — do not merge. Stacked on ainfra-2968-adopt-the-out-of-repo-keystore-convention-in-wordpress (#23261) so the step runs against exactly the code under review. It is closed once it has answered.

Why the PR pipeline can't answer this

#23261 changes where the release signing config finds its keystore, and nothing on a PR exercises that path:

• Prototype builds are the only jobs that assemble an app, and PROTOTYPE_BUILD_TYPE = 'Debug'.
• Lint and the merged-manifest diffs never sign anything.
• release-builds.yml and beta-builds.yml are API-triggered from release automation and checkout_release_branch, so they never see a PR's code.

The first run that would notice is the scheduled trunk-internal build — after merge.

The debug side is worse than uncovered. When the shared keystore is missing, AGP falls back to ~/.android/debug.keystore and the build stays green, so a passing prototype build has never been evidence that the renamed debug.keystore was read.

What to read

One job: 🔐 Validate signing config. Every step is expected to pass; there is no intentional failure here. It fails the build on any unmet assertion, and the three signingReport outputs are attached as artifacts.

What each assertion proves

1. configure_apply on a Buildkite agent writes both keystores into the out-of-repo secrets directory under their new names.
2. wordpressRelease and jetpackRelease report Config: release with Store: pointing at that directory, and print a certificate — so path, store password and key alias all resolve on CI.
3. wordpressDebug and jetpackDebug read the renamed debug.keystore rather than falling back.
4. With the credentials still readable but no upload keystore, both release variants drop to Config: none — the keystore file is what gates signing, so assertion 2 could have gone red.
5. With no shared debug keystore, the debug variants silently fall back to ~/.android/debug.keystore — the failure mode assertion 3 exists to catch.

Both negative cases run against a throwaway user.home holding symlinks to the real credentials, so the agent's secrets directory is never mutated and no decrypted secret is copied anywhere. Each signingReport runs with --no-configuration-cache, since signingConfigs {} is evaluated at configuration time and a reused cache entry would let the negative cases pass without re-reading the disk.

What this still doesn't answer

That AGP can sign and upload a real bundle with this keystore — signingReport proves the config resolves, not that bundleRelease ships. That first happens on the trunk-internal build, which notifies #build-and-ship.

---

Opened by Claude (Opus 5) on behalf of @mokagio with approval.

[dangermattic]

	1 Message
📖	This PR is still a Draft: some checks will be skipped.

Generated by 🚫 Danger

[mokagio]

Answered: the release signing config resolves to the out-of-repo keystore on a Buildkite agent, and both negative cases confirm the file-existence gate is what produces that result. Evidence moved into #23261's description. Closing without merging, as intended.

Posted by Claude (Opus 5) on behalf of @mokagio with approval.

[wpmobilebot]

📲 You can test the changes from this Pull Request in Jetpack Android by scanning the QR code below to install the corresponding build.

	App Name	Jetpack Android
	Build Type	Debug
	Version	pr23266-7dc953e
	Build Number	1498
	Application ID	com.jetpack.android.prealpha
	Commit	7dc953e
	Installation URL	6nl712957qapo

Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

[wpmobilebot]

📲 You can test the changes from this Pull Request in WordPress Android by scanning the QR code below to install the corresponding build.

	App Name	WordPress Android
	Build Type	Debug
	Version	pr23266-7dc953e
	Build Number	1498
	Application ID	org.wordpress.android.prealpha
	Commit	7dc953e
	Installation URL	0mtpfbc7t6d6o

Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.