Fix PackageDeploymentManager test cascade: force-close blocking app on ERROR_PACKAGES_IN_USE retry - #6693
Conversation
…n ERROR_PACKAGES_IN_USE retry Test::Packages::AddPackage() retries AddPackageAsync up to 5 times with exponential backoff (~15s total) when it hits ERROR_PACKAGES_IN_USE (0x80073D02), but always reissues with DeploymentOptions::None. When an earlier test's activated app (e.g. IsPackageRegistrationPending_Pending's Test.PackageManager.M.Black instance) hasn't fully released its package lock within that window -- or a deferred registration from a prior test hasn't completed settling -- every retry fails identically and the in-place upgrade to Test.PackageManager.M.Blacker never makes progress. RemovePackage_Blacker()/RemovePackage_Black() call into this same AddPackage() path whenever the target package is staged-but-not-registered (to register it before removal), and are invoked pervasively across PackageDeploymentManagerTests_Register, _IsPackageRegistrationPending, and _IsReadyOrNewerAvailable -- including from TEST_CLASS_SETUP/TEST_CLASS_CLEANUP. A single stuck AddPackage() there fails the fixture and TAEF cascades every test in the class to [Failed] with empty error text, then the leftover staged package repeats the same race in the next class that runs. Fix: once AddPackageAsync fails with ERROR_PACKAGES_IN_USE, escalate to DeploymentOptions::ForceTargetApplicationShutdown on the next retry so the deployment service force-closes the blocking process itself instead of us guessing how long to wait -- this is the option the platform documents for exactly this error. The first attempt is left as DeploymentOptions::None so a clean install is never destructive. This should eliminate the 0x80073D02 cascade behind ~221 failures in Test-Foundation (PackageDeploymentManagerTests_Register, _IsPackageRegistrationPending, _IsReadyOrNewerAvailable). It does not change the unrelated, already-known IsPackageRegistrationPending_Pending API-behavior assertion at IsPackageRegistrationPending.cpp:108. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7a243429-2b18-4b9a-aea2-f207cc2ea908
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Verified: fix confirmed via targeted rebuild-stage validationValidated via a minimal, targeted run — no full monobuild triggered. Used the monobuild's built-in single-stage rebuild mechanism (
Result: 228 → 14 failures (-214), zero cascade signal remaining
The remaining 14 failures are exactly the two known, out-of-scope, real product-behavior tests called out in scope (
|
Add a short note to the AddPackage() comment clarifying that the escalation can only ever close a process left over from an earlier, already-finished test - no test installs the package it's actively exercising via this helper while that app is running (confirmed: the IsPackageRegistrationPending "_Pending" tests exercise the in-use/deferred scenario through AddPackageByUri's DeferRegistrationWhenPackagesAreInUse path, which this change doesn't touch) - and that unconditional use of ForceApplicationShutdown in Shared.cpp's and TestSetupAndTeardownHelper.h's InstallPackage() helpers is existing precedent for this exact mitigation in this test suite. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7a243429-2b18-4b9a-aea2-f207cc2ea908
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). 1 pipeline(s) were filtered out due to trigger conditions. |
2 participants
Problem
221 of the ~228 failures on WinAppSDK-Test-Foundation (def 192441) stem from one test-isolation defect:
Reference runs:
Root cause
Test::Packages::AddPackage() already retries AddPackageAsync up to 5 times with exponential backoff (~15s total) when it hits ERROR_PACKAGES_IN_USE (x80073D02), but always reissues with DeploymentOptions::None.
RemovePackage_Blacker() / RemovePackage_Black() call into this exact AddPackage() path whenever the target package is staged-but-not-registered (they register it first so it can be removed). These helpers are called pervasively across PackageDeploymentManagerTests_Register, _IsPackageRegistrationPending, and _IsReadyOrNewerAvailable — including from TEST_CLASS_SETUP/TEST_CLASS_CLEANUP.
PackageDeploymentManagerTests_IsPackageRegistrationPending's *_Pending tests activate Test.PackageManager.M.Black (same package family as Blacker) and defer-register Blacker while it's running. A prior fix (#6678) added a wait for the activated process to exit, but the OS-side "in use" bookkeeping for the package family doesn't necessarily clear the instant the process handle signals — the deferred registration completing is a separate async step. When it hasn't settled by the time the next RemovePackage_Blacker() call fires (in the same class's cleanup, or the next class's setup), AddPackageAsync fails with x80073D02 again, and 5 retries at ~15s total isn't always enough. A single stuck call inside a TEST_CLASS_SETUP/TEST_CLASS_CLEANUP fixture fails the whole fixture, and TAEF cascades every test in the class to [Failed] with empty error text — and the leftover staged package repeats the same race in the next class that runs, which is why the cascade spans three classes.
Confirmed directly from the ADO test results for build 154915868 (IsPackageRegistrationPendingForUser_NotInstalled, _Registered, _Pending, and both IsReadyOrNewerAvailable failures all show the identical AddPackageAsync('...Blacker...') = 0x80073D02 ... Test.PackageManager.M.Black_1.2.3.4... failure at WindowsAppRuntime.Test.Package.h:459; all 38 Register::* tests fail with empty error text from the cascaded ClassSetup).
Fix
Once AddPackageAsync fails with ERROR_PACKAGES_IN_USE, escalate to DeploymentOptions::ForceTargetApplicationShutdown on the next retry so the deployment service force-closes the blocking process itself instead of guessing how long to wait — this is the option the platform documents for exactly this error ("If this package is currently in use, the processes associated with the package are shut down forcibly so that registration can continue."). The first attempt is left as DeploymentOptions::None so a clean install is never destructive; only a confirmed ERROR_PACKAGES_IN_USE failure escalates subsequent retries.
This is a single, universal choke point: every RemovePackage_Blacker()/RemovePackage_Black() call across all three affected test classes funnels through this same AddPackage() function, so the fix addresses the cascade regardless of which specific test left the package family "in use".
Scope
Verification plan
Will validate via a targeted rebuild-stage run of WinAppSDK-Test-Foundation (def 192441) and diff the failure list against build 154915868