CI: ghcr image for apt deps + intensive wolfSSL/ibmswtpm2 build caching#534
Open
aidangarske wants to merge 8 commits into
Open
CI: ghcr image for apt deps + intensive wolfSSL/ibmswtpm2 build caching#534aidangarske wants to merge 8 commits into
aidangarske wants to merge 8 commits into
Conversation
Kill the intermittent apt-get mirror timeouts and rebuild waste in CI. apt deps (wolfBoot model): - Add .github/docker/wolftpm-ci/Dockerfile baking the common CI apt deps (autoconf, automake, build-essential, clang, clang-tools, cmake, git, libtool, libtss2-tcti-mssim0, pkg-config, tpm2-tools, valgrind, ...). - Add publish-ci-image.yml to build/push ghcr.io/<owner>/wolftpm-ci:v1.0. Image namespace follows the repo owner so a fork publishes and consumes its own image; lowercases the owner for the push tag. - Move the apt-using Linux jobs into `container:` (sanitizer, pqc-examples, release-checks scan_build, cmake-build static-export, wolfssl-versions-pqc, fwtpm-test ibmswtpm/valgrind). unshare --net jobs get SYS_ADMIN/NET_ADMIN/ NET_RAW caps so they can create a netns inside the container. - Add a reusable apt-retry action and use it for the jobs that cannot move into the image (mixed-OS matrices, version-specific compilers, mono, zephyr, CodeQL), giving bounded retries + Acquire timeouts. cached builds: - Add setup-ibmswtpm action (resolve commit -> actions/cache -> build on miss) and replace the 3 clone+make sites; fwtpm uses MAX_LOADED_OBJECTS=7. - Add setup-wolfcose action (cache wolfCOSE source builds, keyed on the wolfSSL SHA) ready for future wolfCOSE coverage. - Add inline wolfSSL build caching (resolve SHA -> actions/cache -> build on miss) to the jobs that still rebuilt it every run: release-checks, multi-compiler, coverity-scan-fixes, and make-test-swtpm old-wolfssl.
Add `credentials:` (github.actor + GITHUB_TOKEN) to every job that runs in the wolftpm-ci container so they pull the image with the workflow token. This lets the image stay private and work immediately for same-owner pushes and PRs, with no manual "make package public" step. External-fork PRs against a canonical repo still need the package public (or fall back to apt), which is a one-time owner action, but same-owner CI no longer requires it.
github.repository_owner is 'wolfSSL' (mixed case) but ghcr image paths are lowercase, so the container jobs failed to pull ghcr.io/wolfSSL/wolftpm-ci. Pin the consumer image refs to the lowercase ghcr.io/wolfssl/wolftpm-ci path.
- wolftpm-ci image was missing libssl-dev (ibmswtpm2 build needs openssl headers) and iproute2 (fwtpm make-check runs ip/ss inside unshare --net, which exit 127 without them); add those plus the openssl CLI. - multi-compiler and zephyr called the local apt-retry action before actions/checkout, so the action files were not on disk yet; check out the repo first.
- Image: add libclang-rt-18-dev so clang -fsanitize=* can link (compiler-rt is only a Recommends of clang and was dropped by --no-install-recommends), and add zip (an example loads /usr/bin/zip as sample data). - ibmswtpm-tpm2tools: start tpm_server in the same step as the test; GitHub reaps a step's background processes between steps in container jobs. - multi-compiler: raise the 5-minute job timeout to 12 so an occasionally slow apt mirror installing a version-specific compiler is not hard-cancelled.
…uirk) sanitizer, pqc-examples and ibmswtpm-tpm2tools run tpm2-tools make-check which includes activatecredential. In a container that fails with TPM_RC_INTEGRITY against both the IBM simulator and wolfTPM's fwtpm — a tpm2-tools/OpenSSL makecredential integrity quirk specific to the container, not a wolfTPM bug (it passes on host runners, as on master). Keep these on the host runner with apt-retry (still no apt-mirror timeout); the rest stay in the ghcr CI image.
dgarske
previously approved these changes
Jun 19, 2026
The image is built and published; remove the bring-up-only branch entry so publish only runs on master/main Dockerfile changes (and workflow_dispatch).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Kills the intermittent
apt-get updatemirror timeouts and the per-runrebuild waste in CI.
apt timeouts (wolfBoot ghcr-image model)
.github/docker/wolftpm-ci/Dockerfilebakes the common CI apt deps; builtand pushed to
ghcr.io/wolfssl/wolftpm-ci:v1.0bypublish-ci-image.yml.container:(sanitizer, pqc-examples,release-checks scan_build, cmake static-export, wolfssl-versions-pqc,
fwtpm ibmswtpm/valgrind).
unshare --netjobs get SYS_ADMIN/NET_ADMIN/NET_RAW caps. Container jobs authenticate to ghcr with the workflow token,
so the image can stay private for same-repo runs (public only needed for
external-fork PRs).
apt-retrycomposite action (bounded retries + Acquire timeouts) for jobsthat cannot containerize: CodeQL, Windows/macOS matrix legs, mono, zephyr,
version-specific compilers.
cached builds
setup-ibmswtpmcaches the IBM SW TPM simulator (3 sites; MAX_LOADED_OBJECTS=7for fwtpm).
setup-wolfcoseready for future wolfCOSE coverage (keyed on the wolfSSL SHA).added to every job that still rebuilt it each run: release-checks,
multi-compiler, coverity-scan-fixes, make-test-swtpm (main + old-wolfssl).
before merge
ci_ghcr_imageentry from thepublish-ci-image.ymlpush trigger (it exists only to build the image from this branch).
wolftpm-cipackage public so external-fork PRs can pull it.