Skip to content

Flaky test: FCS tooltip #r reference-assembly resolution is non-deterministic (Fsx.Bug5073, ResolveFromGAC) #20276

Description

@T-Gro

Description

A couple of #r reference-assembly tooltip tests in tests/FSharp.Compiler.Service.Tests/Tooltip/TooltipTests.Declarations.fs intermittently produce an empty tooltip in the FCS test host, so the assertReferenceTooltipContains assertions flake:

  • Fsx.Bug5073#r "System", expects the tooltip to contain Reference Assemblies\Microsoft
  • Fsx.HashR_QuickInfo.ResolveFromGAC#r "CustomMarshalers", expects .NETFramework / CustomMarshalers.dll

The root cause looks like non-deterministic reference-assembly / GAC name resolution inside the FCS test host rather than a real product regression (the same #r resolves fine in VS/dotnet fsi).

Current state

  • ResolveFromGAC is already skipped on main with [<Fact(Skip = "…non-deterministic in the FCS test host…")>].
  • Fsx.Bug5073 was briefly skipped in PR [main] Update dependencies from dotnet/roslyn #20100 (the maestro Roslyn dependency bump) to keep that PR green, but the skip was reverted so the flake isn't hidden inside an unrelated dependency update. This issue tracks it instead.

Repro steps

  1. Run dotnet test tests/FSharp.Compiler.Service.Tests --filter "FullyQualifiedName~Fsx.Bug5073" (Windows/Desktop; these are FactForDESKTOP).
  2. Re-run several times — the tooltip is occasionally empty and the assertion fails.

Expected behavior

The reference tooltip for #r "System" / #r "CustomMarshalers" resolves deterministically and the tests pass reliably.

Actual behavior

Intermittently empty tooltip → assertReferenceTooltipContains fails.

Suggested fix

Make the test host's reference-assembly/GAC resolution deterministic (or harden the tooltip resolution / add a retry-until-ready in the test helper), then re-enable both tests.

Known workarounds

Skip the tests (undesirable — hides the flake).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status
    New

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions