Add replay PBT CI proof of concept

[nubtron]

Summary

This PR adds a replay-PBT CI proof of concept for cached integration replay testing.

Highlights:

• Rename compare-check concepts from old/new to record/replay while retaining old flag aliases.
• Add replay-PBT fixture/target refs and cache-only probing.
• Add a small branch-gated PR smoke job for KrakenD replay-PBT.
• Repurpose the existing manual zz-test-worker-poc.yaml workflow as a parallel Replay PBT POC runner.
• Add a replay-PBT matrix script with sharding and fail-loud truncation protection.

CI POC behavior

The manual POC workflow supports:

• changed, all-cached, and all-declared modes
• optional cache seeding
• smoke vs all property sets
• per-target cache restore/save
• sharding to avoid the GitHub 256-job matrix limit
• summary collection as JSON/TSV plus a GitHub step summary table

Security review notes

Before opening this draft, I reviewed the changed workflows for common workflow security issues:

• No pull_request_target usage.
• All external actions in changed workflow paths are pinned to full commit SHAs.
• The POC workflow uses only contents: read permissions.
• No GitHub secrets are used by the new POC workflow.
• No github-script, dynamic script execution, eval, or curl-to-shell pattern added.
• Manual ref inputs are validated before use in git fetch / matrix generation.
• Matrix-derived artifact names use a sanitized artifact_slug.
• Matrix target path components are validated in the matrix script before being used in cache paths.
• Matrix truncation is disabled by default; over-large runs fail with shard guidance instead of silently dropping targets.

Local validation

• YAML parse for changed workflows passed.
• replay-pbt-matrix.py compiles.
• all-declared without sharding fails loudly when over max_targets.
• all-declared with shard_count=2 emits 192 targets for shard 0.
• ddev lint/format passed.
• Replay PBT unit/cache tests passed: 17 passed, 10 skipped.
• Local cached KrakenD smoke run passed: 13 passed, 6 skipped.

Notes

This is intentionally a draft POC. The full manual replay-PBT workflow should be exercised first with a tiny capped run before any full seeded or all-property run.

[dd-octo-sts]

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
agent-reqs	Verify check versions match the Agent requirements file	✅
ci	Validate CI configuration and Codecov settings	✅
codeowners	Validate every integration has a CODEOWNERS entry	✅
config	Validate default configuration files against spec.yaml	✅
dep	Verify dependency pins are consistent and Agent-compatible	✅
http	Validate integrations use the HTTP wrapper correctly	✅
imports	Validate check imports do not use deprecated modules	✅
integration-style	Validate check code style conventions	✅
jmx-metrics	Validate JMX metrics definition files and config	✅
labeler	Validate PR labeler config matches integration directories	✅
legacy-signature	Validate no integration uses the legacy Agent check signature	✅
license-headers	Validate Python files have proper license headers	✅
licenses	Validate third-party license attribution list	✅
metadata	Validate metadata.csv metric definitions	✅
models	Validate configuration data models match spec.yaml	✅
openmetrics	Validate OpenMetrics integrations disable the metric limit	✅
package	Validate Python package metadata and naming	✅
qa-label	Validate the pull request declares whether it needs QA for the next Agent release	✅
readmes	Validate README files have required sections	✅
saved-views	Validate saved view JSON file structure and fields	✅
version	Validate version consistency between package and changelog	✅

View full run