Add Gherkin behaviour tests with dummy runtime#1981
Conversation
Introduce defaults.runtime config, a scripted dummy runtime for deterministic E2E validation, and godog-based behaviour tests against live GitHub/GHA. Signed-off-by: Barak Korren <bkorren@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Site previewPreview: https://bddbbd25-site.fullsend-ai.workers.dev Commit: |
|
Closing to reopen from an upstream branch so CI can access repository secrets. |
|
Review skipped — this PR is already closed. The Posted by fullsend post-review check |
Retro: PR #1981 — Wasted review agent run on short-lived draft PRWhat happened: PR #1981 was opened as a draft from a fork at 10:40 UTC by a human contributor (ifireball) using Cursor. A review agent was dispatched 15 seconds later (run 27090168627). The author closed the PR at 10:47 UTC (7 minutes later) to reopen from an upstream branch so CI could access secrets. The review agent continued running until ~10:51 UTC, completing a full Waste: ~11 minutes of review agent compute (tokens + GHA minutes) were spent on a PR that was closed before the review finished. The same content was then re-reviewed on PR #1982. Existing coverage: All improvement opportunities identified are already tracked by open issues:
Recently closed issues #1566 and #887 suggest active work in this area. No new proposals filed — existing issues comprehensively cover the improvements needed. This PR serves as a concrete real-world example validating the priority of those issues. |
Summary
defaults.runtimeorg config and--runtimeinstall flag with sharedruntime.ResolveFromConfig()selection infullsend run.behaviour-results.jsonfor deterministic assertions.e2e/behaviour/with pluggable GitHub/GitHub Actions drivers, triage scenarios, CI job, and ADR/docs.Test plan
go test ./...go test -tags behaviour -c ./e2e/behaviour/...go test -tags e2e -c ./e2e/admin/...make behaviour-testagainst halfsend org pool withGITHUB_TOKENand--runtime dummyorgsE2E_BEHAVIOUR_GITHUB_TOKENsecretMade with Cursor