Skip to content

Flaky: TestGetStatus_ActivationBlockIsOperatorOnly fails whenever the unix timestamp contains "4711" #1206

Description

@Dumbris

What happens

TestGetStatus_ActivationBlockIsOperatorOnly (internal/httpapi/scope_round11_test.go:122) asserts that a scoped caller's /api/v1/status body does not contain the string 4711:

assert.NotContains(t, body, "4711", "exact deployment-wide retrieve_tools count leaked to a scoped caller")

The body also contains a unix timestamp. When that timestamp happens to contain 4711 as a substring, the assertion fails even though nothing leaked.

It is not hypothetical — this is the confirmed cause of a real CI failure

On PR #1204, all five Build Binaries jobs failed simultaneously on this test. The jobs ran between 16:58:17 and 16:58:43 UTC on 2026-09-03. Unix second 1788454711 — which is exactly 16:58:31 UTC that day — ends in 4711.

The correlated failure is explained by the same fact: the five build jobs run concurrently, so they share the wall clock and all fall into the same one-second window.

The test passes locally at any other time, including 40 consecutive runs.

The fixture's own justification is incorrect

scope_round11_test.go:36-38 says:

A count no other field of a /status response can produce: the sibling numbers are unix timestamps and single-digit server counts, so a raw-body "4711" assertion cannot pass for an unrelated reason.

A 10-digit unix timestamp has 7 candidate positions for a given 4-digit substring, so it contains any specific one roughly 0.07% of the time; millisecond and nanosecond timestamps are worse. Over enough CI runs this recurs, and each occurrence takes out every concurrent job at once.

Suggested fix

The raw-body assertion is the right idea — a projection that renamed the key would still be a leak — so it is worth keeping. Options:

  1. Choose a fixture value a timestamp cannot contain, e.g. a non-numeric sentinel like "retrieve-tools-count-4711", or a number with a non-digit neighbour asserted alongside it ("retrieve_tools_calls_24h":4711).
  2. Strip the known-volatile fields before the raw-body check (decode, delete timestamp, re-encode), keeping the key-name assertions on the full body.

Option 1 is smaller and keeps the assertion honest.

Related

Found while verifying CI for #1204 (Spec 058 mcp-go v1.0.0 bump). The failure is unrelated to that change — internal/httpapi is untouched by that branch.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/build-ciGitHub Actions, Makefile, build scriptskind/testAdding or improving testspriority/mediumImportant but not blocking a release

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions