Skip to content

fix(hubs): make ADF trigger startTime Z-suffix conditional on UTC fallback - #2291

Open
Michael Flanakin (flanakin) wants to merge 8 commits into
flanakin/v15-prepfrom
flanakin/2157-adf-trigger-utc-starttime
Open

fix(hubs): make ADF trigger startTime Z-suffix conditional on UTC fallback#2291
Michael Flanakin (flanakin) wants to merge 8 commits into
flanakin/v15-prepfrom
flanakin/2157-adf-trigger-utc-starttime

Conversation

@flanakin

Copy link
Copy Markdown
Collaborator

Summary

Fixes #2157 — FinOps hub deployments fail in regions not covered by timeZones.bicep's region→Windows-timezone map (e.g. Sweden Central) with InvalidWorkflowTriggerRecurrence.

Root cause: Three ADF schedule triggers hardcode startTime without a trailing Z (UTC zone designator):

  • config_DailySchedule and queries_DailySchedule: '2023-01-01T01:01:00'
  • config_MonthlySchedule: '2023-01-05T01:11:00'

These triggers set timeZone from timeZones.bicep's region lookup, which falls back to the literal 'UTC' for any region not in its explicit map (confirmed: Sweden Central is one such region; likely also Qatar Central, Poland Central, Italy North, Israel Central, though the map should be checked directly rather than assuming this list is exhaustive). Azure Data Factory requires startTime to end in Z (format yyyy-MM-ddTHH:mm:ssZ) whenever timeZone resolves to 'UTC' — otherwise trigger activation fails with InvalidWorkflowTriggerRecurrence, which surfaces during deployment as Init-DataFactory.ps1's Start-AzDataFactoryV2Trigger retrying up to MAX_RETRIES = 20 and then failing the whole deployment.

Fix: Make the Z suffix conditional on the resolved timeZone actually being 'UTC':

startTime: timeZones.outputs.Timezone == 'UTC' ? '2023-01-01T01:01:00Z' : '2023-01-01T01:01:00'

Applied to all three triggers:

  • src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicepconfig_DailySchedule and config_MonthlySchedule
  • src/templates/finops-hub/modules/Microsoft.FinOpsHubs/IngestionQueries/app.bicepqueries_DailySchedule

This preserves the existing local-wall-clock scheduling behavior for every region that is mapped (no Z, non-UTC timeZone), and only adds Z for the fallback case, fixing activation there.

Explicitly not done, and why:

  • Did not unconditionally append Z to all three literals — that would silently switch every mapped region from local-time scheduling to UTC wall-clock scheduling, a user-facing behavior change for currently-working regions.
  • Did not expand the region→timezone map to reduce how often the UTC fallback triggers — that's a separate, non-durable maintenance treadmill, not a fix for this defect.

Out of scope: The issue reporter also mentioned a failure in Norway East. Norway East is present in timeZones.bicep's map ('W. Europe Standard Time', non-UTC), so the root cause above does not explain a Norway East failure. This looks like a second, distinct issue that needs more diagnostic detail from the reporter — it is not addressed by this PR.

Test plan

  • bicep build on both changed files completes cleanly (only the expected @validate() experimental-feature warning, unrelated to this change)
  • Confirmed compiled ARM JSON emits the conditional if(equals(...Timezone..., 'UTC'), '...Z', '...') expression for all three startTime properties
  • pwsh -Command "./src/scripts/Test-PowerShell.ps1 -Lint" — 3585/3585 tests passed
  • Manual validation: deploy to an unmapped region (e.g. Sweden Central) and confirm triggers activate without InvalidWorkflowTriggerRecurrence

🤖 Generated with Claude Code

…nifest (#2260)

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
…storage in private mode (#2259)

Private-mode deployments failed at the first deployment script with
DeploymentScriptStorageAccountWithServiceEndpointEnabled, because the script
storage account was reached through a Storage service endpoint while private
subnets set defaultOutboundAccess to false.

Azure Policy 7bca8353-aa3b-429b-904a-9229c4385837 ("Subnets should be private")
requires defaultOutboundAccess to be present and false, so that setting has to
stay. The service endpoint is the optional half of the conflict and is replaced
here with the documented Azure Files private endpoint path:

- Private subnets keep defaultOutboundAccess: false; the NAT Gateway continues to
  provide egress for the script and Data Explorer subnets.
- The Storage service endpoint is removed from the script subnet.
- The script storage account moves to publicNetworkAccess: Disabled with no
  virtual network rules, reached through a file private endpoint and a
  privatelink.file.core.windows.net zone.
- HubsPrivateNetworking.Tests.ps1 pins all of the above in the compiled ARM.
- Stale defaultOutboundAccess wording removed from six user-facing strings.

Validated live in both affected modes (storage-only, westus): deployment
succeeds, all 7 deployment scripts execute, all three subnets carry
defaultOutboundAccess: false in private mode, the script storage has no virtual
network rules and no public access, and the file private DNS zone is present.

Fixes #2258
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: MSBrett <MSBrett@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ates (#2261)

Co-authored-by: Roland Krummenacher <roland.krummenacher@alescent.com>
Co-authored-by: MSBrett <MSBrett@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
… type (#2251)

Co-authored-by: Roland Krummenacher <roland.krummenacher@alescent.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: MSBrett <MSBrett@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…lback

FinOps hub deployments to regions not covered by timeZones.bicep's
region-to-Windows-timezone map (e.g. Sweden Central) fail with
InvalidWorkflowTriggerRecurrence. Data Factory requires startTime to end
in 'Z' when timeZone resolves to 'UTC', but the three schedule triggers
(config_DailySchedule, config_MonthlySchedule, queries_DailySchedule)
hardcoded startTime without a trailing 'Z', which is only valid when
timeZone is non-UTC.

Emit the 'Z' suffix only when timeZones.outputs.Timezone resolves to the
'UTC' fallback, so mapped, non-UTC regions keep scheduling on local
wall-clock time as before.

Fixes #2157

Note: the reporter also saw a failure in Norway East, which IS in the
timezone map (non-UTC), so this fix does not explain or address that -
it needs separate diagnostic info and is treated as a distinct,
unverified issue.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs: Review 👀 PR that is ready to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants