Skip to content

cli's required Installer (shell) check can be blocked by an unretried apt-get, and reports it as an installer timeout #534

Description

@LukasWodka

Installer (shell) is a required status check on cli's develop, and its first action is an unretried, untimed apt-get. A package-mirror stall therefore blocks every PR in the repo for the full 10-minute job budget, and reports as a test failure.

Measured

Hit on cli#533 — a workflow-only diff that cannot touch installer behaviour at all:

job 96126585157   Installer (shell)   failure   10m16s
  15:34  Set up job
  15:34  Run actions/checkout
  15:34  shellcheck + dash parse      <-- 10 minutes here, then killed
  15:44  Post Run actions/checkout

The step (build.yml:50-59) begins:

- name: shellcheck + dash parse
  run: |
    sudo apt-get update -qq && sudo apt-get install -y -qq shellcheck dash

Nothing after the apt-get line ever ran. The same job on develop at 12:25 the same day completed in 0.3 minutes — so this is a ~30× outlier caused by package fetching, not by the shell checks it exists to run.

It failed twice consecutively on the same head, which is what moves this from "flake, re-run it" to something worth fixing.

Why it matters more than an ordinary flake

  • It is a required check. A mirror blip is indistinguishable from a real installer regression on the PR page, and it blocks the merge either way.
  • The failure names the wrong thing. The annotation is The job has exceeded the maximum execution time of 10m0s on a job called Installer (shell). Whoever sees it reasonably concludes the installer is slow or hanging. Nothing points at apt.
  • The 10-minute budget is spent before the actual work starts. The shell checks themselves take seconds; the timeout is sized for the job, so an apt stall consumes all of it.

Options

  1. Stop installing shellcheck at all. ubuntu-latest images ship shellcheck preinstalled — worth confirming for the current image, and if so this step needs no apt at all. dash likewise is present on Ubuntu by default (/bin/dash is the system sh). If both hold, the fix is deleting the line, which is also the fastest.
  2. Retry with a bound, if something genuinely must be installed: a short timeout around apt-get plus two retries, so a blip costs seconds rather than the job.
  3. Give the step its own timeout-minutes so an apt stall fails fast and legibly instead of consuming the job clock.

(1) is the one I would try first, precisely because it removes the dependency rather than making it more resilient — and a check that installs nothing cannot stall on a mirror.

Not urgent, but not nothing

No data is at risk and nothing ships wrongly. What it costs is that cli's required gate has an unbounded external dependency in its first line, and the failure it produces points at the wrong subsystem.

Found on: cli#533 (the backend#2157 sweep) · job 96126585157

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions