mcpp-language-server 0.0.4: typing an import no longer freezes the editor, the status says whose problem it is, import is colored - #22
Merged
Conversation
…t the end of its line clangd 23.1 spins forever on `import hello.` (or `export module a.`) when nothing but white space or a comment follows the dot on that line, and every later version of the file queues behind the spin: typing any dotted import went through that text, so the editor froze each time. Such a directive is always an error, and clangd reports the same line with `;` right after the dot at once, so that is the text it is given; its diagnostics are mapped back to the editor's text, and a unit opened from disk is rewritten the same way. The rewrite is WA-CLANGD-001 in a new registry of what the server does only because of a defect in clangd (mcppls.engine.clangd.workarounds): each entry names the versions that need it, the upstream defect, the evidence, when it can go and the canary that will say so. The engine's traits are now read from it, the four existing compensations are registered as WA-CLANGD-002 to 005, and the log and the report list the ones a clangd version needs.
…angd restarted, even while the user keeps typing The guards of 0.0.2 and 0.0.3 took a busy clangd for a compiling one: the stuck watch only acts on a clangd that uses no CPU, and a file was not set aside for 120 s after any edit to it, so a spin started by an edit, with the user still typing to fix it, was never acted on, and even once typing stopped recovery took minutes and could hand the same text back. Now a file's main-file build is given a budget from its own history: five times its last build, never under 20 s; past it, with a version or a request waiting on it, clangd will not finish it, busy or not. The file goes to mcppls's engine with the text clangd spun on remembered, and clangd is restarted at once, past the restart cap if need be, since a clangd left spinning answers nothing and holds a core; the file goes back the moment its text is any other, and the same text never does. An edit to the file itself now counts as a rebuild for 10 s, not 120 s; a change to a module it imports keeps the long patience. The file-quarantined issue names the files. `--disable-workaround` turns a registered workaround off, which is how the spin is reproduced on purpose.
…o module can have is never planned With autosave, each half-typed import reached the plan: `import hello` got an empty stand-in module and a rewritten engine database, and a build tool's scan of `import hello.` even got one for the module `hello.`. A name nothing provides, imported by a file the editor changed within the last five seconds, now waits for its stand-in until the file is quiet, and the workspace plans again then; a unit that provides a module still gets its stand-in at once, since building it with an import it cannot resolve is what stalls clangd. Names that are not dotted identifiers with at most one partition are dropped from what a unit imports. Plan issues now say whose problem they are: an unresolved import and a module that does not build are the code's, a missing SDK or toolchain the environment's.
…ver that lost a feature A missing `;` or an import still being typed turned the status to "Some features are limited": any plan issue made the state degraded, so a typo, a slow rebuild and a clangd that lost the file all looked alike, and with autosave the warning flickered as the user typed. Every status issue now carries a category -- code, engine, environment or project -- and only the categories other than code make the state degraded: an import nothing provides and a module that does not compile are the code's and are told where they are, as diagnostics; a missing SDK, an untrusted workspace or a clangd that cannot run are the environment's. A change from ready to degraded goes out only once it has lasted three seconds, so a file set aside and handed back as the user types never reaches the editor; error goes out at once. Conformance gets the checks that prove the import-hang plan on every platform: `type-text` types a line one key at a time and requires every step to be answered in time, `clangd-check` is a workaround's canary, and the fixtures typing-import, typing-import-spin (the real spin, with WA-CLANGD-001 turned off) and workaround-canaries run in CI; module-faults and failure-at-base now expect ready, naming their code issues.
…e stopped for you
The Neovim plugin now asks the server for module-syntax semantic tokens
(initializationOptions.semanticTokens = { modules, moduleType = true }), where the new
semantic_tokens_modules setup() option (default true) feeds `modules`; a user's own
init_options.semanticTokens, if set, replaces the plugin's default outright, and
conflictArbitration stays forced afterwards as before.
@lsp.type.module is linked to @module, and @lsp.type.keyword to @Keyword, both as Neovim
`default` links so a colorscheme or the user's own nvim_set_hl wins; they are set at setup
and put back on every ColorScheme, since colorschemes clear existing links when they load.
@lsp.type.keyword already links to @Keyword on Neovim 0.10 through 0.12 on its own (checked
on 0.10.4 and 0.12.5 locally), so this is mostly a safety net for @lsp.type.module, the
custom type this plugin's moduleType = true asks the server to send.
A new disable_conflicting option (default false, unchanged behavior) has the plugin stop a
clangd or ccls client itself when it attaches to a buffer mcppls also serves, instead of
only saying so once: a client serving only such buffers is stopped outright, one also
serving another buffer is only detached from this one, and either way the plugin says,
once per client, which one it stopped. The default notice now mentions the option. The
stop/detach is deferred one tick (vim.schedule): Neovim's own Client:on_attach() re-sets
attached_buffers[bufnr] right after firing the same LspAttach event, so detaching
synchronously from inside that handler would be undone as soon as it returns.
editors/nvim/tests/smoke.lua covers all of this: the init options plugin sends and how a
user's own init_options and semantic_tokens_modules interact; the highlight defaults and
that they don't override a user's own; and both disable_conflicting branches (stop and
detach-only), against the existing in-process fake clangd/ccls pattern. README documents
the new options and the highlight groups, with how to recolor them.
Ran editors/nvim/tests/smoke.lua (setup and enable modes) locally on Neovim 0.12.5 and
0.10.4; all checks pass on both. CI also runs 0.11.5, which was not available to test here,
but nothing in this change uses APIs newer than 0.10.
… --disable-workaround
… limited, and let people manage other C++ extensions Three pieces of the 0.0.4 import-hang-status-highlight design (§6, §7, §9, §10), scoped to editors/vscode: the server-side halves (issue categories, module semantic tokens) are separate, parallel tracks. - An injected grammar (syntaxes/mcppls-modules.tmLanguage.json, WA-VSCODE-001 in the new src/workarounds.ts registry) colors module, import, export and module names on every keystroke, with no server involved, because VS Code's own cpp grammar defines the rule for this but never uses it. It accepts a name that is still being typed (import hello. colors import and hello) and never matches import or module used as an ordinary identifier. - package.json declares the semanticTokenTypes/Modifiers/Scopes contributions and the mcppls.semanticTokens.modules setting the S3 extension's semantic-token contract needs, and the client now sends initializationOptions.semanticTokens accordingly, so the server's own module semantic tokens (a parallel track) have somewhere to land. - The status bar no longer says the generic "Some features are limited": src/statusText.ts (kept free of vscode, like serverLog.ts, for unit testing) picks the first non-code issue's message instead, shortened for the bar and in full in the tooltip/detail, and skips any issue categorized "code" (the user's own text being wrong, which becomes a diagnostic instead). An issue with no category -- an older server -- is treated as non-code, as every issue was before this field existed. The error state keeps its sentence and appends why. - mcppls.turnOffOtherCppFeatures and mcppls.restoreOtherCppFeatures (src/conflicts.ts, src/conflictCandidates.ts) let a person turn other C++ extensions' language features off, or put them back, in this workspace or everywhere, at any time -- not only through the one-time question. A conflict becoming active again after activation (reinstalled, re-enabled, or its setting turned back on) gets a non-modal notice, once per conflict per session. ccls and any other extension with no enable setting can only be pointed at, in the Extensions view. Nothing here changes another extension's settings without the user choosing it, and every path is reversible.
…us wording and the conflict commands Unit (mocha, plain Node, no VS Code): - test/unit/statusText.test.ts: category filtering, the shortened-vs-full wording rules, and the removal of the generic "Some features are limited" phrase. - test/unit/conflictCandidates.test.ts: which extensions count as active conflicts, and the restore snapshot (an unset override is remembered as undefined, not as a default value). End to end (main suite): - test/suite/grammar.test.ts: opens a scratch .cpp file and reads back _workbench.captureSyntaxTokens for every form the grammar covers, including import hello. still being typed, and the two required negatives (x = import;, obj.import(1);, int module = 5;). - test/suite/workaroundCanary.test.ts: WA-VSCODE-001's canary -- fails, naming the workaround, the day VS Code's own cpp grammar includes its module_import rule from anywhere. (That rule ships under a hash-prefixed repository key, e.g. d9bc4796b0b_module_import, not the bare name the design doc found in the unmangled upstream source, so the check matches by suffix.) - test/suite/semanticTokens.test.ts: decodes the server's semantic tokens and checks the keyword and module types on the fixture's import lines. mcppls 0.0.3's legend has neither yet (confirmed live), so every assertion here skips, not fails, until the parallel server track lands them. Conflicts scenario (test/suite-conflicts): re-enabling clangd.enable after the first-run answer disabled it gets a non-modal notice; mcppls.turnOffOtherCppFeatures (workspace scope, substituted the same way as the conflictAnswer prompt) turns it back off; mcppls.restoreOtherCppFeatures puts back exactly the value from right before that turn-off (true, what the test set it to) rather than the very first answer's value, and removes cpptools's override entirely since it never had one.
…the new conflict commands README: a Highlighting section explaining the two layers and how to customize the color, the new mcppls.semanticTokens.modules and detectConflicts wording in the settings table, the two new commands, and the conflicts section covering turn off/restore at any time, the re-check notice, and that an extension with no enable setting can only be pointed at.
…d Neovim's disable_conflicting
clangd 23.1 has no keyword type and tokenizes no import/module/export line at all (measured 2026-09-25); its own legend also has duplicate names (variable appears twice). mcppls.orchestrator.tokens is a new, pure module: a fixed base legend (the LSP standard types and modifiers, clangd's own extras, and mcppls's own module/partition), a mapping from a core engine's own token-type indices and modifier bits into it by name (any other name the core engine declares is appended, and duplicates collapse to the first entry with that name), the LSP relative encode/decode, and a merge where the core engine's tokens win every position they cover and native tokens only fill the gaps. Covered by tests/test_tokens.cpp: legend construction and its indices, mapping with duplicates and an appended name, the encode/decode round trip (sorted, relative), and the merge (overlap dropped, disjoint gap kept, null only when neither engine answered).
…ntax, complete or not project::scan_syntax_tokens is a new tokenizer, next to scan_source's lexer: export, module and import keywords, and the module or partition name that follows each, from the text alone -- even a name still being typed (import hello. keeps its keyword and moduleName tokens; module :private and a plain module; produce only the keyword). Unlike scan_source it needs no terminating ';', and a dotted name never crosses a line, so a name cut off by a trailing dot or the end of a line still gives a valid token. The native engine now declares textDocument/semanticTokens/full and /range (merge role, gated by a new TokenOptions.modules) and answers with those tokens: module names as the custom `module` type with a `declaration` modifier for the unit's own declaration and a `partition` modifier on partitions, or as `namespace` with no partition modifier when TokenOptions.moduleType is off -- the fallback for a client that never asked for the custom type. engine::Host gets a new semantic_tokens_changed() hook (default no-op, so nothing else implementing Host needs to change) for the workspace's coalesced workspace/semanticTokens/refresh, wired up in the next commit. Covered by tests/test_scan.cpp (every form in the design doc, including the incomplete and pathological ones, and that no token spans two lines) and a new case in tests/test_server.cpp (moduleType on/off, modules=false, range filtering, and a merge through routing::merge_results).
…and refreshes them on change
merge_capabilities now advertises semanticTokensProvider from the server's own legend
(full: true, range: true, no delta), built from whatever the core engine declared --
never appended to the core engine's own legend, so a restart, a --clangd override or no
core engine at all cannot shift an index a client has already seen. routing::merge_results
merges textDocument/semanticTokens/full and /range through tokens::merge, kept pure: the
workspace remaps the core engine's raw answer into the server's legend right where it
arrives (route_client_request), before it ever reaches merge_results, so routing itself
only ever combines two answers already in the same index space. A full/delta request is
rewritten to full before routing sees it, since this server hands out no resultId a delta
could build on.
initializationOptions.semanticTokens ({modules, moduleType}) is read in session.cpp and
carried on SessionOptions to the native engine's factory in cli/options.cpp, the same path
compiler/kit/engine already take.
Workspace gains semantic_tokens_changed(): coalesced to at most one
workspace/semanticTokens/refresh every ~500ms, sent only once initialize is answered and
only when the client declared workspace.semanticTokens.refreshSupport. clangd calls it at
the one place that already tracks every set-aside and hand-back (update_quarantine_issue_),
at its own handshake, and when module preparation goes idle -- three one-line calls, no
other change to clangd.cpp, since another track is changing that file in parallel.
… types
The runner decodes textDocument/semanticTokens/full (or /range, with "range") with the
legend initialize gave, and passes when every entry of "expect"
({"line", "text", "type", "modifiers"?}) is among the decoded tokens. A scenario's own
"initialization-options" object is now merged into the runner's initializationOptions, so
a fixture can ask for something no --client profile sends.
inferred (with the real bundled clangd) asks for {"semanticTokens": {"moduleType": true}}
and checks a declaration's module and partition names come back as the custom `module`
type with `declaration`/`partition` modifiers, merged with clangd, which contributes
nothing on those lines. engine-none (no core engine) keeps the default and checks the
`namespace` fallback instead, so both forms in the design doc are exercised.
conformance/README.md's kind table documents the new kind and the new scenario field.
…answers it, and refresh is a request clangd 23.1 declares no semantic-tokens range request, so advertising `range` had VS Code send ones clangd rejected with "method not found"; `range` is now advertised only when the core engine declares it too (mcppls's own engine alone always answers it). `workspace/semanticTokens/refresh` is a request in LSP, not a notification: it goes out with an id, and its empty answer is dropped like the watcher registrations'. The grammar test's probe imported a partition by its qualified name after `import std;`, which is ill-formed and which clangd 23.1 (and main at 510126255) never finishes under a project's module command: the probe, opened in a real editor, held the session in `preparing` for two minutes. It now imports `:part`, the form partitions are imported by, and is closed once its tokens are read.
…td, which clangd 23.1 never finishes otherwise
…ond clangd 23.1 defect
…clangd at once and remembers the text Review found that when the timeouts of a file's requests set it aside before its spin was found, the spin was ignored: the restart that setting-aside had put off stayed put off, and without the text clangd spun on remembered, the end of the file's term could hand that same text back. A spin found on a file set aside for anything else now records the text and restarts clangd at once, past the cap if need be, the same as one found first. A build tool's scan naming `export module hello.` no longer plans a provider of `hello.` either.
…nstead of spinning, and the checks say so CI found the first defect is a crash on Windows (exit 0x80000003) rather than a spin: the canary took the quick exit for a fixed clangd, and typing-import-spin waited for an engine-spin event that cannot come there. The canary now counts a crash as the defect still being there, a check can name the operating systems it holds on (`only-on`), and typing-import-spin checks the crash restart on Windows and the spin elsewhere. WA-CLANGD-001 keeps clangd from crashing there too.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Typing an
importfroze the editor. clangd 23.1 never finishes a file in which a module name ends in.at the end of its line (import hello.,export module a.), and typing any dotted import passes through that text. The status also said "Some features are limited" for a missing;, andimportwas not colored. Plan, measurements and implementation record:.agents/docs/2026-09-25-import-hang-status-highlight.md; release notes: CHANGELOG [0.0.4].Engine
;right after the dot, which it reports at once as the error it is. Its diagnostics are mapped back to the editor's text. clangd 22.1.8 and 24.0.0git (510126255) do not have the defect; the likely upstream fix is6dcfc17b1b, not yet bisected.src/engine/clangd/workarounds.cpp), with versions, upstream, removal condition and canary. The engine traits come from it,mcppls reportlists the ones in use, and--disable-workaroundturns one off.Status
category(code,engine,environment,project; S3-4-10 to S3-4-14). Only non-code issues make a rootdegraded: a problem in the user's code is a diagnostic where it is.degradedis held for 3 s (S3-4-15).Editors
keyword, plusmoduleor anamespacefallback (S3-6.1). The legend is the server's own, andrangeis advertised only where clangd answers it.mcppls.semanticTokens.modulesand the commands Turn Off / Restore Other C++ Language Features, and names a conflict that appears later.semantic_tokens_modules, a default@lsp.type.modulelink anddisable_conflicting.Tests
type-textandclangd-check, and the fixturestyping-import,typing-import-spin(the real spin, with the workaround off) andworkaround-canaries, on every platform.module-faultsandfailure-at-basenow expectready, with their issues categorizedcode.check alland the spec validator: 0 failures;typing-import,typing-import-spin,workaround-canaries,inferred,engine-none,module-faults,untrustedandfailure-at-basepass.Found along the way (known limit)
import std;comes before anexport importof something nothing provides, under a command that names std's unit.