Catch the smoke suite up so the strict gate runs green - #277
Open
jeremy wants to merge 2 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Updates smoke tests to reflect current authentication, command-group, and posting-move behavior.
Tip
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.
Changes:
- Assert cookie-based
auth tokenrefusal. - Validate bare
boxhelp output. - Create and restore a seen posting for move coverage.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
tests/smoke/auth_test.go |
Updates cookie-auth token expectations. |
tests/smoke/boxes_test.go |
Updates box help and move setup/cleanup. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Three tests went stale against this week's CLI changes, and one skip was the dataset's fault rather than the server's: - auth token now refuses to print a browser session cookie as a bearer token, and the smoke suite authenticates with exactly that cookie. The refusal is the command's behavior under this auth method, so assert it (exit 3, the cookie message) instead of expecting a token. - box became a command group, so bare "hey box" shows help and exits 0. Assert the help names "hey box view" instead of expecting a failure. - The move test skipped forever waiting for a seen posting the seed data never has. When none exists it marks one seen itself and restores the unread state after the move-back cleanup has brought the posting home. With these, HEY_SMOKE_STRICT=1 make test-smoke passes against a current haystack dev server: 156 runs, no skips, no failures.
cliEnv passed the caller's environment through, and hey auth token answers HEY_TOKEN before it looks at stored credentials — so a developer or CI runner with a token exported would run the whole suite as that token instead of the cookie TestMain stored, and the auth token tests would pass for the wrong reason. Filter it once, in cliEnv, so every invocation is isolated rather than just the two tests that noticed.
jeremy
force-pushed
the
strict-smoke-gate
branch
from
August 23, 2026 02:13
5725a6e to
f3997d9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The strict smoke gate (
HEY_SMOKE_STRICT=1 make test-smoke) had never been run. Running it today against a current haystack dev server surfaced three stale tests and one structural skip — all test-side; no CLI behavior changed.TestAuthToken/TestAuthTokenStored—hey auth tokenrefuses to print a browser session cookie as a bearer token since 45b0263, and the smoke suite authenticates with exactly that cookie (auth login --cookieinTestMain). The refusal is the command's behavior under this suite's auth method, so the tests now assert it: exit 3 and the cookie message. The happy path can't be smoke-tested while the harness logs in by cookie.TestBoxNoArgument—boxbecame a command group in Unify resource commands under singular nouns #276, so barehey boxshows help and exits 0. The test now asserts the help nameshey box viewinstead of expecting a failure.TestMovePosting— skipped forever waiting for a seen posting the seed data never provides (it runs beforeTestSeenUnseenmarks anything seen). When no seen posting exists it now marks one seen itself and restores the unread state in a cleanup registered before the move-back cleanup, so LIFO ordering brings the posting home to the Imbox before it goes unseen.Gate result with these fixes, against haystack
1e966b5e2d(current master — the checkout needed updating: collections/Screener/journal-feed/calendar-period JSON all landed server-side this week):make test-smoke: PASS, 156 runs, 0 skipsHEY_SMOKE_STRICT=1 make test-smoke: PASS, 135s, 0 skips, 0 failuresSummary by cubic
Aligns the smoke suite with current CLI behavior and hardens the test harness so the strict smoke gate runs green. No CLI behavior changed; all updates are test-side and env-side.
HEY_TOKEN, which could mask cookie auth and produce false positives; now filtersHEY_TOKENfrom the test env so the suite always uses the stored cookie.Result: both make test-smoke and HEY_SMOKE_STRICT=1 make test-smoke pass against a current haystack dev server (156 runs, no skips, no failures).
Written for commit f3997d9. Summary will update on new commits.