Skip to content

Catch the smoke suite up so the strict gate runs green - #277

Open
jeremy wants to merge 2 commits into
mainfrom
strict-smoke-gate
Open

Catch the smoke suite up so the strict gate runs green#277
jeremy wants to merge 2 commits into
mainfrom
strict-smoke-gate

Conversation

@jeremy

@jeremy jeremy commented Aug 22, 2026

Copy link
Copy Markdown
Member

The strict smoke gate (HEY_SMOKE_STRICT=1 make test-smoke) had never been run. Running it today against a current haystack dev server surfaced three stale tests and one structural skip — all test-side; no CLI behavior changed.

  • TestAuthToken / TestAuthTokenStoredhey auth token refuses to print a browser session cookie as a bearer token since 45b0263, and the smoke suite authenticates with exactly that cookie (auth login --cookie in TestMain). The refusal is the command's behavior under this suite's auth method, so the tests now assert it: exit 3 and the cookie message. The happy path can't be smoke-tested while the harness logs in by cookie.
  • TestBoxNoArgumentbox became a command group in Unify resource commands under singular nouns #276, so bare hey box shows help and exits 0. The test now asserts the help names hey box view instead of expecting a failure.
  • TestMovePosting — skipped forever waiting for a seen posting the seed data never provides (it runs before TestSeenUnseen marks anything seen). When no seen posting exists it now marks one seen itself and restores the unread state in a cleanup registered before the move-back cleanup, so LIFO ordering brings the posting home to the Imbox before it goes unseen.

Gate result with these fixes, against haystack 1e966b5e2d (current master — the checkout needed updating: collections/Screener/journal-feed/calendar-period JSON all landed server-side this week):

  • baseline make test-smoke: PASS, 156 runs, 0 skips
  • HEY_SMOKE_STRICT=1 make test-smoke: PASS, 135s, 0 skips, 0 failures

Summary by cubic

Aligns the smoke suite with current CLI behavior and hardens the test harness so the strict smoke gate runs green. No CLI behavior changed; all updates are test-side and env-side.

  • Auth token: previously expected a printed bearer token; now asserts refusal under cookie auth (exit 3 with “browser session cookie”).
  • Box: previously expected bare “hey box” to fail; now expects help output (exit 0) and mentions “hey box view”.
  • Move: previously skipped when no seen posting; now marks one seen, moves it, then restores unseen after the move-back cleanup.
  • Env isolation: previously inherited HEY_TOKEN, which could mask cookie auth and produce false positives; now filters HEY_TOKEN from the test env so the suite always uses the stored cookie.

Result: both make test-smoke and HEY_SMOKE_STRICT=1 make test-smoke pass against a current haystack dev server (156 runs, no skips, no failures).

Written for commit f3997d9. Summary will update on new commits.

Review in cubic

@jeremy
jeremy requested a review from a team as a code owner August 22, 2026 20:17
Copilot AI balanced review requested due to automatic review settings August 22, 2026 20:17
@github-actions github-actions Bot added the tests label Aug 22, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates smoke tests to reflect current authentication, command-group, and posting-move behavior.

Tip

If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

Changes:

  • Assert cookie-based auth token refusal.
  • Validate bare box help output.
  • Create and restore a seen posting for move coverage.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
tests/smoke/auth_test.go Updates cookie-auth token expectations.
tests/smoke/boxes_test.go Updates box help and move setup/cleanup.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/smoke/auth_test.go
jeremy added 2 commits August 22, 2026 19:09
Three tests went stale against this week's CLI changes, and one skip was
the dataset's fault rather than the server's:

- auth token now refuses to print a browser session cookie as a bearer
  token, and the smoke suite authenticates with exactly that cookie. The
  refusal is the command's behavior under this auth method, so assert it
  (exit 3, the cookie message) instead of expecting a token.
- box became a command group, so bare "hey box" shows help and exits 0.
  Assert the help names "hey box view" instead of expecting a failure.
- The move test skipped forever waiting for a seen posting the seed data
  never has. When none exists it marks one seen itself and restores the
  unread state after the move-back cleanup has brought the posting home.

With these, HEY_SMOKE_STRICT=1 make test-smoke passes against a current
haystack dev server: 156 runs, no skips, no failures.
cliEnv passed the caller's environment through, and hey auth token answers
HEY_TOKEN before it looks at stored credentials — so a developer or CI
runner with a token exported would run the whole suite as that token
instead of the cookie TestMain stored, and the auth token tests would pass
for the wrong reason. Filter it once, in cliEnv, so every invocation is
isolated rather than just the two tests that noticed.
@jeremy
jeremy force-pushed the strict-smoke-gate branch from 5725a6e to f3997d9 Compare August 23, 2026 02:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants