fix(hubs): repair ListCost/ContractedCost for rows without a meter/offer ID

[Michael Flanakin (flanakin)]

Summary

Fixes defects 2 and 4 from #2214 (defect 1 is #2248; defect 3 is out of scope — see below):

• Defect 2 — cost-repair fallback unreachable without a meter/offer ID. IngestionSetup_v1_0.kql and IngestionSetup_v1_2.kql gated the ListCost/ContractedCost repair fallback behind the same condition as the price-sheet lookup, which requires x_SkuMeterId/x_SkuOfferId. Rows without either — most commonly third-party Marketplace/ISV purchases, which have no Microsoft retail list price by design — never reached the repair fallback, leaving ListCost at 0 despite a real EffectiveCost. That collapses x_TotalSavings and corrupts Effective Savings Rate reporting (negative ESR observed on production hubs in Rows with EffectiveCost but no ListCost produce negative savings rates in the ADX dashboard #2214; reported again independently after an EA→MCA migration in Wrong ESR Calculation after moving from EA to MCA in Rate Optimization #2235).
• Defect 4 — MissingListCost flag doesn't clear after repair. The flag in x_SourceChanges was computed before the repair ran, so a successfully repaired row still reported itself as broken (98.9% of flagged rows on one production hub, per Rows with EffectiveCost but no ListCost produce negative savings rates in the ADX dashboard #2214's data).

How the fix works

Splits the combined gate into two: tmp_MissingCost (no meter/offer ID requirement) now gates entry into the repair fallback; tmp_MissingPrices narrows that further to rows the price-sheet join can actually resolve a key for. A row that's tmp_MissingCost but not tmp_MissingPrices skips the join — all joined columns come back null — and falls through the existing case() blocks' null-safe branches straight to the EffectiveCost-based fallback. That's the same outcome #2214 asked for, and it needed zero new branches in the case() blocks — they already handle the join's no-match case correctly, just weren't reachable for these rows before.

MissingListCost is cleared from x_SourceChanges once ListCost is known-good post-repair, in both files.

Scope notes (defect 3 excluded)

v1.2 already has x_SourceValues/checkReal provenance tracking, and it automatically picks up newly-repaired rows — old_ListCost is captured before the gate runs, so checkReal('ListCost', old_ListCost, ListCost) downstream correctly detects the change. No new code needed.

v1.0's Costs_final_v1_0 schema does not get x_SourceValues and this is intentional, not deferred: v1.0 is kept specifically so people can revert to legacy behavior, and its schema must never gain new columns. Confirmed via a Pester test that asserts x_SourceValues never appears in IngestionSetup_v1_0.kql.

Test plan

• Added HubsMissingCostGate.Tests.ps1 — asserts both KQL files have the gate split, the join is correctly scoped, the merge-back uses the right gate, the MissingListCost clearing logic is present, and v1.0's schema stays frozen
• Added MissingCostGateSplit.kql — executable equivalence harness (same pattern as ContractedCostTolerance.kql) covering: no meter/offer ID, only one of the two IDs set, unused spend commitments, non-Microsoft providers, already-correct prices, no effective price. Verified by hand against the gate logic (PASS = 0 rows)
• Full PowerShell lint suite passes (3586/3586)
• Full unit test suite passes (2295/2295)
• Maintainer review — draft because this touches the ingestion pipeline's write path; want a second pass on the case() fallback behavior for previously-unreachable rows before merging

Related: #2214, #2235, #2286

🤖 Generated with Claude Code