Skip to content

Add direct snprintf vs ctprintf benchmark comparison page - #8

Open
embtom with Copilot wants to merge 1 commit into
mainfrom
copilot/more-direct-comparison-snprintf-ctprintf
Open

embtom with Copilot wants to merge 1 commit into
mainfrom
copilot/more-direct-comparison-snprintf-ctprintf

Conversation

Copilot AI commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

The benchmark dashboard on gh-pages only plots each benchmark's history on its own chart. It doesn't put snprintf and ctprintf next to each other. This PR adds a side-by-side comparison generated from the same benchmark run.

  • scripts/benchmark-comparison.py (new)

    • Reads the Google Benchmark JSON and pairs benchmark_snprintf_<case> with benchmark_ctprintf_<case>.
    • For each pair it reports CPU time for both and the ratio (Nx faster or Nx slower for ctprintf).
    • Outputs a static comparison.html with paired bars, plus a markdown table.
    • If the run has repetitions, it uses the mean aggregate and ignores the other aggregates.
    • A new case shows up automatically if it follows the same benchmark_{snprintf,ctprintf}_<case> naming. Unpaired benchmarks are skipped.
  • .github/workflows/benchmark.yml

    • Appends the table to $GITHUB_STEP_SUMMARY.
    • Commits comparison.html to the root of gh-pages, next to the existing history dashboard. The page links back to that dashboard.
    • The commit and push step runs after github-action-benchmark, which also pushes to gh-pages. The existing concurrency group serializes runs, so I don't expect the two pushes to race.
  • README.md: links to the comparison page.

The benchmark sources and the existing dashboard are unchanged.

I only ran the script locally against a small hand-written results file. The workflow steps and the gh-pages publish haven't been exercised in CI yet.

Co-authored-by: embtom <16663460+embtom@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants