This repository was archived by the owner on Sep 14, 2026. It is now read-only.
forked from ast-grep/langs
-
Notifications
You must be signed in to change notification settings - Fork 1
feat(cmake): add pinned native grammar with five-platform qualification #27
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
3007648
feat(cmake): add pinned native grammar and platform qualification
harjotgill a18d77f
fix(cmake): preserve pinned vendor bytes on Windows checkout
harjotgill eed4fc2
test(cmake): diagnose Windows malformed-input AST discrepancy
harjotgill c66844b
test(cmake): recognize missing-token syntax recovery
harjotgill File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| name: Qualify CMake native package | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| pull_request: | ||
| paths: | ||
| - 'packages/cmake/**' | ||
| - 'pnpm-lock.yaml' | ||
| - '.github/workflows/cmake-qualification.yml' | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| qualify: | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| - os: ubuntu-24.04 | ||
| arch: x64 | ||
| - os: ubuntu-24.04-arm | ||
| arch: arm64 | ||
| - os: macos-15-intel | ||
| arch: x64 | ||
| - os: macos-15 | ||
| arch: arm64 | ||
| - os: windows-2025 | ||
| arch: x64 | ||
| runs-on: ${{ matrix.os }} | ||
| timeout-minutes: 15 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: '24.15.0' | ||
| architecture: ${{ matrix.arch }} | ||
| - uses: pnpm/action-setup@v4 | ||
| with: | ||
| version: '9.15.9' | ||
| - name: Verify runtime architecture | ||
| run: node -e "if(process.arch !== '${{ matrix.arch }}') process.exit(1)" | ||
| - name: Install locked parser runtime | ||
| run: pnpm install --filter @coderabbitai/ast-grep-lang-cmake --frozen-lockfile --ignore-scripts | ||
| - name: Prepare pinned grammar sources | ||
| run: node packages/cmake/source.cjs | ||
| - name: Build Unix library | ||
| if: runner.os != 'Windows' | ||
| run: node packages/cmake/build.cjs | ||
| - name: Build Windows library | ||
| if: runner.os == 'Windows' | ||
| shell: cmd | ||
| run: | | ||
| for /f "usebackq tokens=*" %%i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath`) do set "VSINSTALL=%%i" | ||
| call "%VSINSTALL%\VC\Auxiliary\Build\vcvars64.bat" | ||
| if errorlevel 1 exit /b 1 | ||
| node packages/cmake/build.cjs | ||
| - name: Test native parser | ||
| run: node packages/cmake/test.cjs | ||
| - name: Relocate binary to installed-package layout | ||
| run: node -e "const fs=require('node:fs'); const p='packages/cmake/prebuilds/prebuild-${{ runner.os }}-${{ runner.arch }}'; fs.mkdirSync(p,{recursive:true}); fs.renameSync('packages/cmake/parser.so',p+'/parser.so')" | ||
| - name: Test without build-tree binary | ||
| run: node packages/cmake/test.cjs | ||
| - uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: prebuild-${{ runner.os }}-${{ runner.arch }} | ||
| path: packages/cmake/prebuilds/prebuild-${{ runner.os }}-${{ runner.arch }}/parser.so | ||
| if-no-files-found: error | ||
| retention-days: 7 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # Preserve the exact upstream bytes authenticated by UPSTREAM.json on Windows. | ||
| vendor/** -text |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| The MIT License (MIT) | ||
|
|
||
| Copyright (c) 2025 Uy Ha | ||
|
|
||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
|
|
||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
|
|
||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # CMake grammar for ast-grep | ||
|
|
||
| This package provides the native `tree_sitter_cmake` language registration. | ||
| It parses source syntax; it does not execute CMake, resolve dependencies, or | ||
| establish which build configuration is active. | ||
|
|
||
| Local build/load checks pass on Linux x64/ARM64 and macOS ARM64. A cross-compiled | ||
| macOS x64 binary also passes runtime tests under Rosetta; this is not native | ||
| Intel CI qualification. Windows is not yet tested. Do not release this draft | ||
| until the complete platform matrix passes. There is no runtime download, | ||
| install script, or compiler fallback. | ||
|
|
||
| ## Source and builds | ||
|
|
||
| `vendor/` contains unchanged generated C sources from the MIT-licensed | ||
| `tree-sitter-cmake` 0.7.2.post1 Python distribution. `UPSTREAM.json` records the | ||
| provenance and exact file hashes. `pnpm source` verifies every vendored file | ||
| before preparing the ignored `src/` build directory. No Python runtime or | ||
| regeneration step is involved. | ||
|
|
||
| `pnpm build` compiles with the maintainer machine's C compiler; `pnpm test` | ||
| loads the result through ast-grep. The native build must run and pass separately | ||
| on each release target. Place the resulting library in | ||
| `prebuilds/prebuild-<platform>-<architecture>/parser.so`, matching the existing | ||
| ast-grep language-package layout. | ||
|
|
||
| `pnpm pack` requires all five targets already covered by the shared grammar | ||
| packages: Linux ARM64/x64, Windows x64, and macOS ARM64/x64. Because the shared | ||
| registry initializes every grammar, shipping fewer targets could break unrelated | ||
| language parsing. Windows is missing, so packing is currently expected to fail. | ||
| File presence is only a packaging guard, not proof of successful qualification. | ||
| Published contents exclude | ||
| the build-tree `parser.so`, generated sources, and development scripts. Test the | ||
| extracted package on every target before publishing. No package is published by | ||
| the source/build/test commands. | ||
|
|
||
| `.github/workflows/cmake-qualification.yml` builds and tests five explicit | ||
| targets, including relocation to the installed-package layout. It does not | ||
| publish packages and has not yet run. Windows builds require the Visual Studio | ||
| x64 developer environment. The existing upstream-delegating publish workflow | ||
| is not this package's release path. Shared `@coderabbitai/ast-grep-langs` | ||
| integration is a separate dependent change. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "repository": "https://github.com/uyha/tree-sitter-cmake", | ||
| "distribution": "https://pypi.org/project/tree-sitter-cmake/0.7.2.post1/", | ||
| "version": "0.7.2.post1", | ||
| "files": { | ||
| "parser.c": "71843d5d805b60d2a1f16c56dd886f00da66e265e578e3226b4975a6b1a30784", | ||
| "scanner.c": "6f8973c8091994cf6b8eea8c8d88cc931464bc64c8f3dc7e3226db9973ac8270", | ||
| "tree_sitter/alloc.h": "b29c1c9fb7cc82f58c84b376df1297d6e2737a1d655fd356db0859e3c29c2fea", | ||
| "tree_sitter/array.h": "5bdf6ed1a78e3409fd443e085ca967a64c188a5d082aaf7f819bccd53a471c94", | ||
| "tree_sitter/parser.h": "180b893c8734778fd32f372dfbc27bd6ad1cd2221f26150b31256ff6716320d2" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| const { spawnSync } = require('node:child_process') | ||
| const { join } = require('node:path') | ||
|
|
||
| if ( | ||
| !['darwin', 'linux', 'win32'].includes(process.platform) || | ||
| !['arm64', 'x64'].includes(process.arch) || | ||
| (process.platform === 'win32' && process.arch !== 'x64') | ||
| ) { | ||
| throw new Error( | ||
| `Unsupported CMake native build target: ${process.platform}/${process.arch}`, | ||
| ) | ||
| } | ||
| // Compile the pinned generated C; do not regenerate a grammar from another | ||
| // revision or fetch tools/sources at build time. This runs only for maintainers. | ||
| const windows = process.platform === 'win32' | ||
| const args = windows | ||
| ? [ | ||
| '/nologo', | ||
| '/LD', | ||
| '/O2', | ||
| '/std:c11', | ||
| `/I${join(__dirname, 'src')}`, | ||
| join(__dirname, 'src/parser.c'), | ||
| join(__dirname, 'src/scanner.c'), | ||
| `/Fe:${join(__dirname, 'parser.so')}`, | ||
| ] | ||
| : [ | ||
| process.platform === 'darwin' ? '-dynamiclib' : '-shared', | ||
| // A Rosetta Node process still invokes the host compiler. Select the target | ||
| // explicitly so the produced library matches the runtime loading it. | ||
| ...(process.platform === 'darwin' | ||
| ? ['-arch', process.arch === 'x64' ? 'x86_64' : process.arch] | ||
| : []), | ||
| '-fPIC', | ||
| '-O2', | ||
| '-I', | ||
| join(__dirname, 'src'), | ||
| join(__dirname, 'src/parser.c'), | ||
| join(__dirname, 'src/scanner.c'), | ||
| '-o', | ||
| join(__dirname, 'parser.so'), | ||
| ] | ||
| // Windows runs in the Visual Studio developer environment. No tool download or | ||
| // runtime compilation fallback; qualification must actually load the result. | ||
| const result = spawnSync(windows ? 'cl.exe' : 'cc', args, { | ||
| cwd: __dirname, | ||
| stdio: 'inherit', | ||
| }) | ||
| if (result.error) throw result.error | ||
| if (result.status !== 0) | ||
| throw new Error(`CMake compiler failed: ${result.status}`) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| const assert = require('node:assert/strict') | ||
| const { existsSync } = require('node:fs') | ||
| const { join } = require('node:path') | ||
|
|
||
| // A local parser.so is deliberately not part of the published package. Never | ||
| // publish a native package that only works on its maintainer's build machine. | ||
| for (const target of [ | ||
| 'Linux-ARM64', | ||
| 'Linux-X64', | ||
| 'Windows-X64', | ||
| 'macOS-ARM64', | ||
| 'macOS-X64', | ||
| ]) { | ||
| assert( | ||
| existsSync(join(__dirname, 'prebuilds', `prebuild-${target}`, 'parser.so')), | ||
| `Missing qualified prebuild: ${target}`, | ||
| ) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| declare const registration: { | ||
| readonly libraryPath: string | ||
| readonly extensions: string[] | ||
| readonly languageSymbol: string | ||
| } | ||
| export = registration | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| const { existsSync } = require('node:fs') | ||
| const { join } = require('node:path') | ||
|
|
||
| const platforms = { darwin: 'macOS', linux: 'Linux', win32: 'Windows' } | ||
| const architectures = { arm64: 'ARM64', x64: 'X64' } | ||
| const local = join(__dirname, 'parser.so') | ||
|
|
||
| module.exports = { | ||
| libraryPath: existsSync(local) | ||
| ? local | ||
| : join( | ||
| __dirname, | ||
| 'prebuilds', | ||
| `prebuild-${platforms[process.platform]}-${architectures[process.arch]}`, | ||
| 'parser.so', | ||
| ), | ||
| extensions: ['cmake'], | ||
| languageSymbol: 'tree_sitter_cmake', | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| { | ||
| "name": "@coderabbitai/ast-grep-lang-cmake", | ||
| "version": "0.0.1", | ||
| "description": "Pinned CMake tree-sitter grammar for ast-grep", | ||
| "license": "MIT", | ||
| "main": "index.js", | ||
| "types": "index.d.ts", | ||
| "files": ["index.js", "index.d.ts", "LICENSE", "UPSTREAM.json", "prebuilds"], | ||
| "scripts": { | ||
| "source": "node source.cjs", | ||
| "build": "node source.cjs && node build.cjs", | ||
| "test": "node test.cjs", | ||
| "prepack": "node check-package.cjs" | ||
| }, | ||
| "devDependencies": { | ||
| "@ast-grep/napi": "0.45.3" | ||
| }, | ||
| "publishConfig": { | ||
| "access": "public", | ||
| "registry": "https://registry.npmjs.org/" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| const { createHash } = require('node:crypto') | ||
| const { readFileSync, mkdirSync, copyFileSync } = require('node:fs') | ||
| const { join, dirname } = require('node:path') | ||
| const upstream = require('./UPSTREAM.json') | ||
|
|
||
| // Offline build preparation: checked-in upstream bytes, never a network fetch. | ||
| for (const [file, expected] of Object.entries(upstream.files)) { | ||
| const source = join(__dirname, 'vendor', file) | ||
| const actual = createHash('sha256').update(readFileSync(source)).digest('hex') | ||
| if (actual !== expected) throw new Error(`Upstream source mismatch: ${file}`) | ||
| const target = join(__dirname, 'src', file) | ||
| mkdirSync(dirname(target), { recursive: true }) | ||
| copyFileSync(source, target) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| const assert = require('node:assert/strict') | ||
| const { parse, registerDynamicLanguage } = require('@ast-grep/napi') | ||
| const registration = require('./index.js') | ||
| registerDynamicLanguage({ CMake: registration }) | ||
|
|
||
| const root = source => parse('CMake', source).root() | ||
| const commands = source => | ||
| root(source).findAll({ rule: { kind: 'normal_command' } }) | ||
| assert.equal( | ||
| commands('FetchContent_Declare(Catch2 GIT_TAG v3.5.4)')[0].kind(), | ||
| 'normal_command', | ||
| ) | ||
| assert.equal( | ||
| commands('# FetchContent_Declare(Fake)\n#[=[\nfind_package(Fake)\n]=]') | ||
| .length, | ||
| 0, | ||
| ) | ||
| assert.equal(commands('set(DOC "find_package(Fake)")').length, 1) | ||
| assert.equal( | ||
| root('find_package(${NAME})').findAll({ rule: { kind: 'variable_ref' } }) | ||
| .length, | ||
| 1, | ||
| ) | ||
| const malformed = root('find_package(') | ||
| const errors = malformed.findAll({ rule: { kind: 'ERROR' } }) | ||
| const pending = [...malformed.children()] | ||
| let missingToken = false | ||
| while (pending.length) { | ||
| const node = pending.pop() | ||
| if (node.text() === '' && !node.isNamed()) missingToken = true | ||
| pending.push(...node.children()) | ||
| } | ||
| if (!errors.length && !missingToken) { | ||
| const describe = node => ({ | ||
| kind: node.kind(), | ||
| text: node.text(), | ||
| children: node.children().map(describe), | ||
| }) | ||
| console.error( | ||
| 'Malformed-input AST diagnostic:', | ||
| JSON.stringify(describe(malformed)), | ||
| ) | ||
| } | ||
| assert( | ||
| errors.length > 0 || missingToken, | ||
| 'Malformed source must retain an error or missing-token recovery marker', | ||
| ) | ||
| console.log(`CMake native parser passed: ${process.platform}/${process.arch}`) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.