Skip to content

fix(tools): walk array item schemas and raise MaxDepth (#71, #82) - #97

Merged
juemerson-at-purestorage merged 5 commits into
dmann000:integration/capability-map-2026-08from
juemerson-at-purestorage:fix/issue-71-82-schema-walk-depth-arrays
Aug 5, 2026
Merged

fix(tools): walk array item schemas and raise MaxDepth (#71, #82)#97
juemerson-at-purestorage merged 5 commits into
dmann000:integration/capability-map-2026-08from
juemerson-at-purestorage:fix/issue-71-82-schema-walk-depth-arrays

Conversation

@juemerson-at-purestorage

Copy link
Copy Markdown
Collaborator

Two bugs in the same schema-walking helper in tools/lib/PfbSpecTools.ps1, landing together
because they interact. Fixes #71. Closes #82's map and reporting scope.

Targets the integration/capability-map-2026-08 branch, not main — see Sequencing below.

The two bugs

#71MaxDepth defaults to 8, truncating fb2.12–2.16 allOf chains. Five body fields
on PATCH /password-policies carried introducedVersion = 2.17 when the specs say 2.16.
Runtime-visible, not cosmetic: Assert-PfbApiCapability reads bodyProperties, so it would
wrongly refuse those five parameters against an array on REST 2.16. The 2.17 spec
restructuring flattened these chains, which is why the defect self-heals from 2.17 onward
and is invisible against current data.

#82 — the walk never descended an array body's items. Four array-bodied endpoints
recorded no usable bodyProperties, hiding 23 fields from both the runtime gate and the
drift report.

They compose: descending through items consumes a depth level, so #82 needs #71's
headroom. Raising MaxDepth alone does nothing for #82, and adding items alone does
nothing for #71.

Why the fix is at the call site, not in the walker

#82's obvious fix — teach Add-PfbSchemaPropertyNodes to descend items — would have
silently corrupted Data/PfbResponseShapeMap.json. That walker is shared with
Get-PfbSpecResponseShapes, whose entire contract is that an envelope's properties and its
items[] element's properties are two deliberately-separate levels. Teaching the walker
to descend unconditionally collapses them, and the response-shape map's cross-version removal
detection would start comparing incomparable sets, with nothing to warn.

So the items hop happens at the Get-PfbSpecCapabilities call site, matching the precedent
Get-PfbSpecResponseShapes already sets. Add-PfbSchemaPropertyNodes is untouched.

The guard that proves it: Data/PfbResponseShapeMap.json is byte-identical before and
after — 423de668a78356bad13b6d345bf9e834eca3438e7badf96577e78a5bbf93fef6. If that SHA had
moved, the fix leaked.

MaxDepth = 32 is not a guess: Get-PfbSpecResponseShapes already defaults to it, with a
comment documenting the same fb2.12–2.16 truncation measured there as 184 false removals at
depth 8 versus 7 true ones at 32.

The regenerated artifact — exactly seven endpoint records change

Group Change
#71 5 × introducedVersion 2.17 → 2.16 on PATCH /password-policies (name, id, enabled, is_local, location)
#82 23 bodyProperties recovered: POST /nodes/batch +12, POST /resource-accesses/batch +2, PUT /workloads/tags/batch +5, POST /fleets/members/batch +4
#82 readOnlyBodyProperties on POST /nodes/batch resolves from empty to 8 fields: capacity, chassis_serial_number, data_addresses, details, id, raw_capacity, status, unique
2 order-only records, no value changes: PATCH /ssh-certificate-authority-policies (bodyProperties key order) and PATCH /file-systems (top-level key order — contextScope and readOnlyBodyProperties swap position)

Nothing else. One correction to how #82 was described: those four endpoints did not have
an empty bodyProperties. Each had exactly one entry keyed by the empty string — the
nameless artifact of a walk that reached the array node and could not descend it. That is
what the 23 real fields replace.

Verification of the artifact itself

Because there is no CI gate comparing the committed map against generator output
(cross-platform-tests.yml never fetches the gitignored tools/specs/), the artifact was
checked by construction rather than assumed:

  • Deterministic. Three consecutive regenerations produced identical SHA-256.
  • Attributable. Reverting tools/lib/PfbSpecTools.ps1 to the base and regenerating
    reproduces Data/PfbCapabilityMap.json byte-identical to the committed Phase-0
    artifact
    . So every change above is caused by this PR's code — none is drift, staleness,
    or nondeterminism. This also independently confirms the Phase-0 map is exactly what the
    generator produces.
  • Phase 0 undisturbed. schemaVersion still 2, 632 endpoints, and the full
    contextScope cross-tab unchanged: array/default 604, array/declared 1,
    array/live-tested 1, fleet/declared 4, fleet/live-tested 3, unknown/unknown 19.

The Reports/ diff is two independent halves — split across two commits

Conflating them is how the pre-existing half gets mistaken for damage this PR caused, so they
are separated at the commit level and each is independently reviewable.

Commit 28c34f2 — pre-existing, not caused by this PR. Regenerates Reports/ with the
walker held at the base. Reports/PfbApiDriftReport.json on main was one gap stale, which
is why the "nothing vanishes" invariant in Tests/Build-PfbApiDriftReport.Tests.ps1 has been
red locally. Root cause: New-PfbFileSystemReplicaLink's [Nullable[bool]]$RemoteDefaultExports
goes through a conditional assignment the drift tracer cannot follow → confidence high
partial → enrichment disabled → systemicGaps/conventionStrength 252 → 246. Not spec
staleness; analysedVersions is identical at 2.28.

This diff (271/19/14/5 lines across four files) is numerically identical to what
origin/automated/update-api-capability-map already holds — that workflow has failed at its
Open pull request step on every run since 2026-07-24, so main never received it.

Commit e0d9a2b — this PR's own effect. Only three files move:
Data/PfbCapabilityMap.json plus the drift-report pair. PUT /workloads/tags/batch gains 5
missingBodyProperties (copyable, key, namespace, resource, value); addable body
properties 422 → 427. Only one of the four batch endpoints appears, because
Set-PfbWorkloadTag is the only existing cmdlet calling any of them — the other three are
#44's scope. Reports/PfbFieldCmdletMap.json, Reports/PfbValueEnumMap.json and their .md
siblings do not move.

The .gitignore change

tools/ is removed from .gitignore. The rule was inert for already-tracked files,
which is why every tools/ file was committed despite it. Removing it stops new files
under tools/ being silently refused — confirmed during this work: git add on a tracked
tools/ file prints the ignore warning and exits non-zero while still staging the file,
which will short-circuit any && git commit chain.

Live verification — FB-A, REST 2.26

Seven ledger rows, every one stamped
gitRef = fix/issue-71-82-schema-walk-depth-arrays@e0d9a2b, so the code under test is this
branch and not main. Outcomes below are the harness's own outcome field.

Cmdlet Endpoint Outcome What it establishes
New-PfbFileSystem POST /file-systems Pass (421 ms) The body-property gate accepts a normal typed body against the regenerated map
Get-PfbFileSystem GET /file-systems Pass (276 ms) Read-back provisioned = 1073741824 — the create really landed
Update-PfbFileSystem PATCH /file-systems Pass (271 ms) The order-only record. Observed state change 1 GiB → 2 GiB (2147483648), so the key-order shuffle is inert at runtime
Set-PfbWorkloadTag PUT /workloads/tags/batch Fail, HTTP 400 (302 ms) #82's one reachable endpoint — see below
Update-PfbFileSystem PATCH /file-systems Pass (322 ms) Destroy, for cleanup
Remove-PfbFileSystem DELETE /file-systems Pass (290 ms) Eradicate
Get-PfbFileSystem GET /file-systems Fail, HTTP 400 (228 ms) Confirms eradication — "File system does not exist." No residue left

The Set-PfbWorkloadTag failure is the informative result, and it confirms rather than
contradicts the #95 claim below.
The array answered HTTP 400: Workload does not exist.
a rejection of the resource, not of a body field. The call therefore reached the wire and
was never gated client-side: Tags is [hashtable[]], it hits the -is [IDictionary]
guard, and the body-field loop is skipped before bodyProperties is consulted. The 23
recovered fields are inert at runtime today, exactly as stated below — this measures
that rather than asserting it.

PATCH /password-policies could not be live-tested at all, for a second reason on top of
the version ceiling: password-policies is on the harness family deny-list, so the call
returns Skipped-Policy ("write verb 'Update' is denied for family 'password-policies'")
and never reaches the array. Not routed around.

What this does NOT do — stated explicitly

Sequencing

This is the second of the two consecutive regenerations of Data/PfbCapabilityMap.json
agreed in #84.
Phase 0 (#96) went first and took schemaVersion 2; this PR rebased onto it and regenerated
once on the stable base.

Both are stacked on integration/capability-map-2026-08 so the combined state is built
and tested before anything reaches main, rather than the two changes meeting for the first
time after they have both landed. One final PR will bring the integration branch to main.

…tems

Two defects in Get-PfbSpecCapabilities' request-body walk, both changing
Data/PfbCapabilityMap.json and both runtime-visible through
Private/Assert-PfbApiCapability.ps1, which reads bodyProperties.

dmann000#71 -- the two body-schema helper calls passed no -MaxDepth, so both took the
helpers' own default of 8. The fb2.12-2.16 schemas compose through allOf chains
deeper than that, so five PATCH /password-policies fields recorded
introducedVersion 2.17 instead of 2.16 and would have been refused against a
2.16 array. Get-PfbSpecCapabilities now takes -MaxDepth, defaulting to 32 --
the value Get-PfbSpecResponseShapes already uses, chosen there by measuring
this same truncation (184 false removals at 8 versus 7 true ones at 32).

dmann000#82 -- a request body that is itself `type: array` carries its element schema
on the `items` sibling keyword, not as a property, so the walk terminated
immediately and four batch endpoints recorded an empty bodyProperties, hiding
23 fields from both the runtime gate and the drift report.

The items hop is done at the CALL SITE, not by teaching
Add-PfbSchemaPropertyNodes to descend `items`. That walker is shared with
Get-PfbSpecResponseShapes, whose contract is that an envelope's properties and
its items element's properties stay two separate levels; collapsing them would
silently change Data/PfbResponseShapeMap.json and make its cross-version
removal detection compare incomparable sets. This mirrors the hop
Get-PfbSpecResponseShapes already performs at its own call site. Verified:
Data/PfbResponseShapeMap.json regenerates byte-identical.

Both regressions are covered by fixtures confirmed to fail before the fix.

Artifact regeneration is deliberately NOT in this commit -- it is sequenced
behind Fusion Phase 0 per issue dmann000#84.

Refs dmann000#71, dmann000#82. Unblocks dmann000#44.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rule has never had any effect. It was added in 69fe478 -- the same commit
that first committed tools/ -- as a hedge while it was undecided whether the
toolchain should be tracked ("Not yet decided whether this should be tracked").
Ignore rules do not apply to already-tracked files, so all 14 files under
tools/ have been tracked from birth and the rule has been inert ever since.

Removing it is a no-op for the working tree and a small improvement going
forward: a NEW file added under tools/ now shows up in git status instead of
being silently invisible, which is how tooling work here has previously gone
missing.

tools/specs/ keeps its own separate rule (.gitignore:36) and stays ignored --
verified that all 29 cached spec files remain ignored after this change, and
that git surfaces no newly-untracked files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The comment explaining why array bodies go unchecked is invalidated by the
issue dmann000#82 map change in this branch, in both of its halves.

It stated that every array-bodied endpoint carries "bodyProperties": {}, so a
per-element check "could never fire". That is no longer true -- the four batch
endpoints now carry real per-element fields.

It then predicted that "this loop picks it up for free if a future map
representation ever lands." That was wrong even when written: the loop is
guarded on $Body being an IDictionary, and an array body arrives as
[hashtable[]] (Set-PfbWorkloadTag passes -Tags straight through), so the loop
is skipped before the map is consulted. A richer map alone changes nothing.

Replaced with what is actually true now: the map records the fields, the type
guard is the remaining blocker, and relaxing it is a real behaviour change that
can refuse calls which succeed today -- deliberately not smuggled in with a
generator fix. Also records that the blast radius is nil today (only
Set-PfbWorkloadTag reaches such an endpoint, all its fields are 2.23) and that
this stops holding once dmann000#44 adds cmdlets for the other three.

Comment-only. No behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Pre-existing correction, isolated deliberately. This commit reverts nothing
and fixes nothing in the generator -- it regenerates Reports/ with
tools/lib/PfbSpecTools.ps1 held at the Phase-0 base, so the entire diff here
is staleness that already existed on main before this branch.

Reports/PfbApiDriftReport.json on main was one gap behind the committed
capability map, which is why the "nothing vanishes" invariant in
Tests/Build-PfbApiDriftReport.Tests.ps1 has been red locally. Root cause is
New-PfbFileSystemReplicaLink's [Nullable[bool]]$RemoteDefaultExports being
sent through a conditional assignment the drift tracer cannot follow, so
confidence degrades high -> partial, enrichment is disabled, and
systemicGaps/conventionStrength move 252 -> 246.

Not spec staleness: analysedVersions is identical at 2.28.

The diff here (271/19/14/5 lines across four files) is numerically identical
to what origin/automated/update-api-capability-map already holds -- that
workflow has failed at its "Open pull request" step on every run since
2026-07-24, so main never received this regeneration.

Splitting it out means the next commit's diff contains only what this PR's
code change actually causes.
…000#71 and dmann000#82

Everything in this diff is caused by the walker fix in the previous commits.
Verified by construction: reverting tools/lib/PfbSpecTools.ps1 to the base and
regenerating reproduces Data/PfbCapabilityMap.json byte-identical to the
committed Phase-0 artifact, so nothing here is drift or nondeterminism.

Capability map -- exactly 7 endpoint records change, in four groups:

  dmann000#71  5 x introducedVersion 2.17 -> 2.16 on PATCH /password-policies
       (name, id, enabled, is_local, location)
  dmann000#82  23 bodyProperties recovered across the 4 array-bodied endpoints:
       POST /nodes/batch +12, POST /resource-accesses/batch +2,
       PUT /workloads/tags/batch +5, POST /fleets/members/batch +4
  dmann000#82  readOnlyBodyProperties on POST /nodes/batch resolves from empty to 8:
       capacity, chassis_serial_number, data_addresses, details, id,
       raw_capacity, status, unique
  ---  2 order-only records, no value changes:
       PATCH /ssh-certificate-authority-policies (bodyProperties key order)
       PATCH /file-systems (top-level key order; contextScope and
       readOnlyBodyProperties swap position)

Note the four batch endpoints did not have an EMPTY bodyProperties before --
each had exactly one entry keyed by the empty string, the nameless artifact of
a walk that reached the array node and could not descend it. That is what the
23 real fields replace.

The map is deterministic: three consecutive regenerations produced identical
SHA-256.

Data/PfbResponseShapeMap.json is byte-identical
(423de668a78356bad13b6d345bf9e834eca3438e7badf96577e78a5bbf93fef6). This is
the guard that the fix stayed at the Get-PfbSpecCapabilities call site and did
not leak into the shared Add-PfbSchemaPropertyNodes walker, whose contract is
that an envelope and its items element are two separate levels.

Phase 0 is undisturbed: schemaVersion still 2, 632 endpoints, and the full
contextScope scope-x-provenance cross-tab is unchanged at
array/default 604, array/declared 1, array/live-tested 1, fleet/declared 4,
fleet/live-tested 3, unknown/unknown 19.

Drift report: PUT /workloads/tags/batch gains 5 missingBodyProperties
(copyable, key, namespace, resource, value); addable body properties 422 -> 427.
Only this one of the four batch endpoints appears, because Set-PfbWorkloadTag is
the only existing cmdlet calling any of them -- the other three are dmann000#44's scope.

Reports/PfbFieldCmdletMap.json, Reports/PfbValueEnumMap.json and their .md
siblings do not move under this change.
@juemerson-at-purestorage
juemerson-at-purestorage merged commit af8cb52 into dmann000:integration/capability-map-2026-08 Aug 5, 2026
4 checks passed
juemerson-at-purestorage added a commit that referenced this pull request Aug 13, 2026
Integration: Fusion Phase 0 prerequisites + capability-map schema-walk fixes (#96, #97)
juemerson-at-purestorage added a commit that referenced this pull request Aug 13, 2026
…ility-map workflow (#63)

No CI job has ever regenerated Data/PfbCapabilityMap.json or
Data/PfbResponseShapeMap.json and compared the result against what is
committed, so a generator change committed without regenerating passes
silently. That comparison had to be done by hand for #96 and #97.

Hashes the two maps before the build steps overwrite them, then names the
outcome in the run summary. Deliberately reports rather than fails: a
legitimate diff is this workflow's whole purpose, since a newly published REST
version changes both maps, so a hard failure would red the scheduled run every
time one lands. The value is that drift is now stated instead of inferred from
the PR body.

Also retires the third copy of the Pester invocation. It could not be replaced
by a workflow_call to cross-platform-tests.yml -- a reusable workflow runs on a
fresh runner with a fresh checkout, so it would test the committed Data/ and
Reports/ rather than the ones this job just regenerated, which is the entire
reason the step was inline. A script runs in this job's workspace.

Step ordering verified unchanged: the response-shape map is still built after
the spec fetch and before the drift report, per the load-bearing note already
in this file.

Refs #63
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant