Skip to content

[DO NOT MERGE] Testing new delta build flow#17066

Draft
dmcilvaney wants to merge 10 commits intotomls/base/mainfrom
damcilva/test_build
Draft

[DO NOT MERGE] Testing new delta build flow#17066
dmcilvaney wants to merge 10 commits intotomls/base/mainfrom
damcilva/test_build

Conversation

@dmcilvaney
Copy link
Copy Markdown
Contributor

@dmcilvaney dmcilvaney commented May 6, 2026

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

  • The toolchain has been rebuilt successfully (or no changes were made to it)
  • The toolchain/worker package manifests are up-to-date
  • Any updated packages successfully build (or no packages were changed)
  • Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
  • Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
  • All package sources are available
  • cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
  • LICENSE-MAP files are up-to-date (./LICENSES-AND-NOTICES/SPECS/data/licenses.json, ./LICENSES-AND-NOTICES/SPECS/LICENSES-MAP.md, ./LICENSES-AND-NOTICES/SPECS/LICENSE-EXCEPTIONS.PHOTON)
  • All source files have up-to-date hashes in the *.signatures.json files
  • sudo make go-tidy-all and sudo make go-test-coverage pass
  • Documentation has been updated to match any changes to the build system
  • Ready to merge
Summary

What does the PR accomplish, why was it needed?

Change Log
  • Change
  • Change
  • Change
Does this affect the toolchain?

YES/NO

Associated issues
  • #xxxx
Links to CVEs
Test Methodology
  • Pipeline build id: xxxx

@dmcilvaney dmcilvaney force-pushed the damcilva/test_build branch 11 times, most recently from 557a52d to 453ebaa Compare May 8, 2026 01:13
@dmcilvaney
ci(prcheck): use 'azldev component changed' for affected-component de…
fe7fc1c 
…tection

Replace the bash 'git diff | grep /sources' step with structured azldev
commands for lock validation, change detection, and scoped rendering.

Pipeline step order:
  1. Lock check -- 'azldev component update -a -q -O json', fail if any
     component has changed == true. Lock-update JSON published as a
     pipeline artifact for triage.
  2. Changed-component detection -- 'azldev component changed
     --include-unchanged' writes the full per-component JSON to disk,
     published via ob_outputDirectory. The --include-unchanged flag
     ensures the JSON contains every known component, which is needed
     for the renderable-set filter in step 4.
  3. Source/identity consistency tripwire -- hard-fail if any component
     reports sourcesChange == true with a changeType not in the
     allow-list {added, changed, deleted}. Prevents unauthenticated
     rewrites of the rendered 'sources' file under an existing
     component's identity. Data path is severed (subsequent steps
     skip); PR check remains advisory until ADO task 19179 removes
     job-level continueOnError.
  4. Scoped render -- render set is the union of components flagged by
     'azldev component changed' (inputs differ) and components whose
     spec tree was touched directly in the PR (git diff under specs/,
     mapped back to component names by compute_render_set.py). Deleted
     and unknown components are excluded via a renderable-set filter
     built from the full --include-unchanged JSON.
  5. Prcheck API -- switches from --components <csv> to
     --changed-components-file <path>, filtering to entries with
     sourcesChange == true and changeType in {added, changed}
     (allow-list, mirroring the consistency tripwire).

Also:
* Add --changed-components-file flag (mutually exclusive with
  --components) and _load_components_from_file() to run_prcheck.py.
  Uses an allow-list of changeType values for defense-in-depth.
* Add compute_render_set.py for render-set computation.
* Document AZLDEV_ALLOW_ROOT in ADO pipeline instructions (OneBranch
  containers run as root, azldev refuses by default).
* Mark changedComponentsFile pipeline variable as isreadonly=true.
* Switch API_BASE_URL to $(ApiBaseDirectUrl) (bypasses AFD).
@dmcilvaney dmcilvaney force-pushed the damcilva/test_build branch from 5e35cd8 to c873a79 Compare May 8, 2026 05:59
@dmcilvaney dmcilvaney force-pushed the damcilva/test_build branch from c873a79 to 76b54d6 Compare May 8, 2026 19:23
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 8, 2026 •
edited
Loading

📄❌ Rendered specs are out of date

FIX: — run this and commit the result:

azldev component render nano

Or download the fix patch and apply it:

gh run download 25588424331 -R microsoft/azurelinux -n rendered-specs-patch
git apply rendered-specs.patch
Category Count
Content diffs 0
Extra files (untracked) 1
Missing files (deleted) 0

Files to add

These files are produced by azldev component render but are missing from your branch. Add them.

  • specs/n/nano/nanorc

dmcilvaney added 8 commits May 8, 2026 17:57
@dmcilvaney
fixup! ci(prcheck): use 'azldev component changed' for affected-compo…
310132d 
…nent detection
@dmcilvaney
ci(pipeline): add package build to Control Tower integration
2268899 
Add a 'Submit package build to Control Tower' step that calls the
/api/Scenario/package endpoint after the prcheck step succeeds. The
step is gated on PR triggers so unmerged code never kicks off a build.

Naming: the pipeline now does more than source upload, so rename to
reflect what it actually does (call Control Tower).
* sources-upload.yml -> control-tower-integration.yml
* sources-upload-stages.yml -> control-tower-integration-stages.yml
* Stage PRCheck -> Integration
* Job CallControlTowerAPI -> UploadAndBuild

Both API calls live in a single job. Two jobs would have doubled the
fixed-cost OneBranch SDL/binary-analysis injections, required a
cross-job artifact handoff for the changed-components JSON, and bought
us only marginal isolation -- upload finishes in minutes and the
package-build call only briefly polls to confirm acceptance.

The ADO pipeline definition's 'YAML file path' must be updated in the
portal from sources-upload.yml to control-tower-integration.yml when
this lands.

Package-build step details:
* condition: and(succeeded(), ne(Build.Reason, PullRequest))
* Reuses the changed-components JSON from earlier in the job, filters
  to changeType in {added, changed}, submits with packageTarget=azl4,
  isScratchBuild=true.
* run_package_build.py polls briefly (default 5 min) just to confirm
  the job left Queued; full build progress is monitored in CT itself.
@dmcilvaney
DROP BEFORE MERGE: enable end-to-end testing from a draft PR
b870179 
Testing-only commit so we can exercise the full PR-3 flow from a draft
PR against the existing ADO build definition.

Changes:
* Rename control-tower-integration.yml -> sources-upload.yml (+ stages
  template) so the existing ADO pipeline definition keeps firing
  without portal edits.
* Lift the PullRequest skip in run_prcheck.py so prcheck gets called
  from a draft PR run.
* Lift the PullRequest exclusion on the package-build step condition
  so scratch builds get submitted from a draft PR run.

(The earlier API_BASE_URL hardcode is no longer needed: PR-1 now wires
$(ApiBaseDirectUrl) from the variable group, which already points at
APIM directly.)
@dmcilvaney
DO NOT MERGE: words benign-overlay test fixture (buildable)
b1fe7e6 
Adds a Summary tweak via spec-search-replace overlay so 'azldev
component changed' reports words with changeType=changed. The spec
Source: URL and lookaside sources file stay at their upstream values so
Control Tower can actually fetch the tarball and rebuild.

Sources file is unchanged from upstream, so sourcesChange=false and the
prcheck upload path is skipped. The package-build path is the one being
exercised here.

Drop this commit before merging anywhere.
@dmcilvaney
DO NOT MERGE: nano specs-diff test fixture (specs-only edit)
2a78a05 
Pairs with the words benign-overlay fixture to exercise the second
input to compute_render_set.py: a rendered-spec edit with no
.comp.toml change. The file shows up in 'git diff --name-only specs/'
and gets mapped to the 'nano' component; the union with azldev's
component-changed output puts both 'words' and 'nano' in the render
set, visible in the pipeline log.

A pure file-addition (not modify/delete) is the only way to exercise
the path without also tripping check_rendered_specs.py, which only
flags modifications and deletions. Drop before merging anywhere.
@dmcilvaney
DO NOT MERGE: delete cvsps to test --diff-filter=d on deleted components
5069a07 
Removes cvsps component + specs + lock. The specs-diff render set
should NOT include cvsps because --diff-filter=d excludes deleted
paths. azldev component changed should report changeType=deleted
which is also excluded from the render set. Net: cvsps should not
appear in the render set or trigger a render call.

Drop this commit before merging anywhere.
@dmcilvaney
fixup! DO NOT MERGE: nano specs-diff test fixture (specs-only edit)
32b590b 
Remove test fixture file -- no longer needed for pipeline validation.
@dmcilvaney
Add missing nano file
6ea25dd
@dmcilvaney dmcilvaney force-pushed the damcilva/test_build branch from c9303be to 6ea25dd Compare May 9, 2026 01:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ddstreetmicrosoft ddstreetmicrosoft Awaiting requested review from ddstreetmicrosoft ddstreetmicrosoft will be requested when the pull request is marked ready for review ddstreetmicrosoft is a code owner

@reubeno reubeno Awaiting requested review from reubeno reubeno will be requested when the pull request is marked ready for review reubeno is a code owner

@christopherco christopherco Awaiting requested review from christopherco christopherco will be requested when the pull request is marked ready for review christopherco is a code owner

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dmcilvaney