Repository navigation
test: table-drive failure classification on the check, lookup, and LSP surfaces - #1027
Merged
Merged
Conversation
…P surfaces Add one table test per public surface that reports provider or session failures. `check` and `lookup` live in a new wright-driver integration test; LSP diagnostics lives in wright-language. Every row drives its surface end to end and pins the diagnostic `code` plus the exit class (`check`/`lookup`) or the boundary code (`diagnostics`), so a wrapper that re-classifies a typed failure — a refusal mapped to `unavailable`, an `invalid-*` lookup code defaulted to `SourceError`, or a session diagnostic relabeled `provider-refusal-*` — fails its row. Removed the tests the tables subsume: the lookup classification unit test, the two process-level lookup refusal tests, the LPP provider failure-class test, and the refusal/diagnostics halves of two language-service tests (the version-negotiation and rename halves stay). Mutation checks: mapping refusals to `unavailable`, defaulting `invalid-*` lookup codes to `SourceError`, and relabeling a session diagnostic `provider-refusal-invalid-selection` each fail a table. Fixes #1022
Review of the failure-classification tables found two defects: - The lint-rule-error construction rows pinned `SourceError`, but every CLI surface that can emit that diagnostic (wright-cli `run_configured`, `run_lookup`, `serve::run`) discards `Diagnostic::class` and exits USAGE. Per the owner ruling the rows are removed rather than re-pinned; the class/wrapper disagreement is left for a separate owner decision. Each table now notes why the kind is deliberately unpinned. - The test-file headers named a nonexistent `failure_classification` file and claimed every row drives `session.check()`/`Lookup` end to end, though the session-boundary rows only reach `CompilerSession::new`/`ToolService::new`. The headers now name the real tables and describe the boundary each row drives. No production change.
Collaborator
Author
|
Review findings addressed in 7045f4b (test-only; no production change):
Verification: all three tables pass; Ready for follow-up review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds one table test per public surface that reports provider or session failures, per the #1022 owner decision:
crates/wright-driver/tests/check_failures.rs— thecheckenvelope. EveryProviderErrorvariant (injected through anLppSourceProviderover a scriptedLanguageProvider) plus the session failure kinds that reach the surface: theCompilerSession::newconstruction diagnostic (invalid-selection),session-config-changed, and thesource-provider-*/profile-workshop-onlyload refusals. Each row pins the diagnosticcode, theclass, and theexitcode the class resolves to.crates/wright-driver/tests/lookup_failures.rs— the tool-serviceLookupresponse. Connect-channel failures (lpp/initialize) and request-channel failures (lpp/lookup) driven by a scripted provider process, plus the session-boundary failure kinds. Each row pinsToolErrorInfo.codeandclass(or theunavailableverdict). Notes in the file record the failure kinds that cannot reach the surface.crates/wright-language/tests/provider_failures.rs— LSPdiagnosticson an.opydocument. Same connect/request-channel coverage plus the session-construction diagnostics; each row pins the publishedcodeand error severity (this surface carries no exit class).lint-rule-erroris deliberately unpinned on the driver surfaces:Diagnostic.classdeclaresSourceError, but the CLI wrappers that can emit the diagnostic —run_configuredandrun_lookupincrates/wright-cli/src/main.rsandserve::runincrates/wright-cli/src/serve.rs— discarddiagnostic.classand hardcode exitUSAGEfor every session-construction failure. The declared class and the reachable surface behavior disagree; which side is correct is left for a separate owner decision, and this PR makes no production change.Removed the coverage the tables subsume: the
lookup_failure_classunit table,opy_lookup_carries_an_initialize_refusal_with_its_code,opy_lookup_reports_unavailable_when_the_provider_lacks_the_capability,lpp_source_provider_distinguishes_failure_classes, and the refusal/diagnostics halves of twowright-languageservice tests (the version-negotiation and rename halves remain — they cover mechanisms the tables do not). The only production diff is the deletedmod tests;for the removed unit-test file; no behavior change.Verification
cargo test -p wright-driver -p wright-language— all suites pass (the new tables included).cargo test -p wright-cli— all suites pass.cargo test --workspace --all-targets --all-features— pass.cargo clippy --workspace --all-targets --all-features -- -D warnings— clean.cargo fmt --all -- --check— clean.git diff --check— clean.provider_failure_codemapping refusals tounavailable→ all three tables fail.lookup_failure_classwithout theinvalid-*arm →lookuptable fails.connect_opy_providerrelabeling a session diagnostic toprovider-refusal-<code>→ LSP diagnostics table fails.Unverified
cd ui && npm ci && npm run check && npm test && npm run build && npm run check:size) — not applicable; the change is Rust test-only,ui/untouched.provider-iorequest-channel, client session-state errors after a live connect,provider-not-configuredforopyonlookup/LSP) have no deterministic trigger — documented in each table's notes and covered at the layer where they are reachable.