Report firmware sizes from CI

[dhalbert]

Code by Claude, design and review by @dhalbert.

Build .json and .html reports that list each board's firmware sizes.

How

• build_release_files.py records each language build in sizes/<board>.json. The board job uploads it as an artifact named zz-sizes-<board> with one-day retention. The zz- prefix sorts them after the boards in the artifact list. The sizes file records, per language:

  • bytes of flash used
  • size of the flash region
  • whether the build succeeded
  • whether the size was measured, or predicted for a language the translation size prediction skipped

• A new script tools/ci_firmware_sizes.py merges the records into:

  • 0-sizes.json for use by other tools
  • 0-sizes.html, which is a table of every board's free flash for the largest translation build. It has user-sortable columns and filtering. The data is embedded in the page so it can stand alone.

  ci_firmware_sizes.py also adds one line to the job summary giving the number of boards built and pointing at the two reports.

• build-ci fetches the per-board records with a small Node script, tools/ci_download_sizes.mjs, which looks each zz-sizes-<board> up by exact name from the scheduler's board list. actions/download-artifact matches a pattern against a listing of the run's artifacts that stops at 1000, and a full build has about 1350, so the first run of this PR silently reported 486 of 673 boards. The script is JavaScript because the only supported client for the artifact service outside the REST API, which is rate-limited to 1000 requests per hour per repository, is the @actions/artifact library.

• The build-ci job runs the merge after the board jobs and uploads each report as its own artifact, unzipped, so 0-sizes.html and 0-sizes.json open directly from the run's artifact list. On pushes to main and release branches and on published releases it also copies them to s3://adafruit-circuit-python/bin/0-sizes/ as adafruit-circuitpython-sizes-<version>.html and .json, where the versioned names keep a history that the usual version-glob cleanup prunes.

• The report steps use continue-on-error, so a reporting problem cannot fail the required check.

[tannewt]

How big are the size files? Could you use job outputs instead? They are limited to 1MB per job and 50MB overall.

https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idoutputs

[dhalbert]

How big are the size files? Could you use job outputs instead? They are limited to 1MB per job and 50MB overall.

That was considered during the design phase, but artifacts worked out better. Job output consideration summary:

Job outputs: strings capped at 1 MB per job. The blocker is that a matrix job has a single set of outputs shared by every leg, so with 673 board legs the last one to finish overwrites the rest. Output names are also fixed in the YAML, so a leg cannot invent sizes_. And the board job lives in a reusable workflow, which would need to declare workflow-level outputs on top.

[dhalbert]

Claude wrote this: what was verified in Build CI run 35254751381, on commit 2682ba8.

• Board jobs: 673 ran, all succeeded, and the run has one zz-sizes-<board> artifact per board artifact, none missing either way. In the pewpew_lcd log each language build is followed by its "bytes used" line, and each language the size prediction skipped has its "Predicted flash size ... -> skip" line. Its record holds 15 measured languages and 2 predicted ones, with status, source and used as documented.
• Download: the build-ci step logs "Downloaded 673 of 673 size records" with no missing-record annotation. The first run of this PR, which used actions/download-artifact with a pattern, got 486 of 673 because the run's artifact listing stops at 1000.
• Reports: 0-sizes.json and 0-sizes.html appear unzipped in the artifact list. The JSON has 673 boards, 660 with a measurable flash region, 1100 predicted language rows and no failed builds; its run block names the run, commit and version 10.4.0-alpha.2-8-gaa4183954a. The HTML embeds the same 673 rows and its script parses. @dhalbert opened the page from this run and confirmed that the "Free flash" chips filter the table, the port tiles filter by port, and the column headers sort.
• Gate: "Check build results" reported all jobs succeeded, and the job has no warnings or errors of its own. The failure guard was exercised by accident on the previous run, where the download found nothing: the step continued, the annotation appeared, and the gate stayed green.
• S3: both upload steps were skipped by the upload_aws action's own condition, with no aws call in the log. After merge, aws s3 ls s3://adafruit-circuit-python/bin/0-sizes/ should show adafruit-circuitpython-sizes-<version>.html and .json.

[dhalbert]

Ready to review. Click on the 0-sizes.html file in Artifacts to see the board sizes table. Columns are sortable, click on the rectangles to choose different architectures, narrow the size reporting by clicking on the colored rectangles.