Skip to content

Fix intermittent thv install failures in visual tests - #1354

Merged
danbarr merged 2 commits into
mainfrom
fix-visual-thv-install
Nov 17, 2026
Merged

danbarr merged 2 commits into
mainfrom
fix-visual-thv-install

Conversation

@danbarr

@danbarr danbarr commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

Description

The visual regression job has been failing intermittently on the Install ToolHive CLI step (for example, #1353's run). scripts/install-thv.sh exited after about 0.2s with no output, so the failure wasn't diagnosable from the log.

The likely cause is GitHub API rate limiting. The script fetched release metadata without authentication, and anonymous requests share a 60-per-hour limit per IP, which hosted runners regularly use up. Which runner a job landed on decided whether it passed. The error was silent because curl -s hid the HTTP status, and the fallback message was captured by the command substitution instead of reaching the log.

Changes:

  • install-thv.sh sends GITHUB_TOKEN as a bearer token when it's set. Vercel and local runs without a token behave as before.
  • Fetch failures now print curl's HTTP error and the failing URL to stderr.
  • The visual regression and baseline update workflows no longer hardcode ToolHive 0.49.0. They read the version from the toolhive entry in .github/upstream-projects.yaml (currently v0.66.0), so the CLI used for snapshots moves with release sync PRs instead of drifting.

The context7 metadata block is masked in its snapshot. If v0.66.0's thv registry info output has a different number of lines than v0.49.0's, the page height changes and this PR's visual check needs a /update-snapshots.

Type of change

  • Bug fix (typo, broken link, etc.)

Related issues/PRs

Intermittent failures on #1353 and on the add-policy-webhook-docs branch.

🤖 Generated with Claude Code

danbarr and others added 2 commits November 17, 2026 14:19
Anonymous GitHub API calls share a 60/hour per-IP limit that CI
runners exhaust, which failed the visual regression job
intermittently. Send GITHUB_TOKEN when it's set.

The fetch error was also invisible: curl -s hid the HTTP status, and
the fallback message was captured by the command substitution instead
of reaching the log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read the CLI version from the toolhive entry in
.github/upstream-projects.yaml instead of a hardcoded 0.49.0 pin, so
it moves with release sync PRs. Pass the job token to the install
step to avoid API rate limits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs-website Ready Ready Preview Nov 17, 2026 7:21pm UTC

Request Review

Copilot AI balanced review requested due to automatic review settings November 17, 2026 19:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues blocking the installer and visual-test workflow changes.

No open issues 🎉

What changed in this PR

This PR addresses intermittent ToolHive CLI installation failures in visual tests by authenticating release metadata requests and improving error reporting. It also aligns visual-test CLI versions with the documentation’s upstream release configuration.

Changes:

  • Use GITHUB_TOKEN for release metadata requests when available.
  • Report curl errors and the failing API URL to stderr.
  • Replace hardcoded CLI versions in both visual workflows with the version from .github/upstream-projects.yaml.
File Description
scripts/​install-thv.sh Adds optional API authentication and visible fetch-failure diagnostics.
.github/​workflows/​update-visual-baselines.yaml Passes the token and reads the configured ToolHive version for baseline generation.
.github/​workflows/​_visual-regression.yaml Passes the token and reads the same version for visual comparisons.

Review effort: Balanced

@danbarr
danbarr enabled auto-merge (squash) November 17, 2026 19:23
@danbarr
danbarr merged commit 0807f0c into main Nov 17, 2026
8 checks passed
@danbarr
danbarr deleted the fix-visual-thv-install branch November 17, 2026 23:11

This branch was successfully deployed

1 active deployment
Preview — a9603182 Deployed Nov 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants