Add GitHub issue tests with per test reporting#71
Draft
gaurav wants to merge 2 commits intoadd-github-issue-testsfrom
Draft
Add GitHub issue tests with per test reporting#71gaurav wants to merge 2 commits intoadd-github-issue-testsfrom
gaurav wants to merge 2 commits intoadd-github-issue-testsfrom
Conversation
Each BabelTest assertion now gets its own pytest test entry, so individual assertion failures are reported separately instead of being collapsed into a single per-issue result. Achieved by: - Adding `repo_id` to `GitHubIssueTest` so `__str__` no longer triggers lazy PyGitHub API calls to resolve org/repo name. - Adding `repo_id` param to `get_test_issues_from_issue()` and threading it through to every `GitHubIssueTest` constructor. - Adding `GitHubIssuesTestCases.get_all_test_issues()` which returns a flat `list[pytest.param]`, one per assertion (issues with no BabelTest syntax are silently skipped). - Rewriting `test_github_issues.py` to parametrize over `GitHubIssueTest` objects via `get_all_test_issues()`, eliminating the inner loop that hid individual failures. The `--issue` filter is updated to use the stored `repo_id` field. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…I endpoint. Moves parametrization into a pytest_generate_tests hook in a new tests/github_issues/conftest.py. When --issue is provided, calls the new get_specific_test_issues() method which uses the single-issue REST endpoint (1–4 API calls) instead of paginating through all issues. Full runs are unaffected. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a version of PR #67 that uses per-test reporting, i.e. each GitHub issue will appear as a single test result.
I think doing this with subtests will be better.
WIP