diff --git a/CHANGELOG.md b/CHANGELOG.md index dad005ee3..cb651f556 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -466,7 +466,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - In Rust, a call through a value's declared type, like `reader.as_mut().poll_read(cx)` on `reader: Pin<&mut R>`, now reads the declaration in scope at the call. Before, a `let` of the same name inside an earlier block, or a struct literal's field of that name written above the call, could be taken for it. Re-index Rust projects after upgrading. - In C++, every name a declaration lists now has the declaration's type, like `strings_found` in google/breakpad's `ByteBuffer symbols_found, strings_found;` or `data_` in a class's `Table *index_, *data_;`, so calls on it link to that class's methods. Before, only the first name had its type read, and a call on another name linked to nothing or to the class of an earlier variable with the same name. - In C++, a call on a parameter, a local lambda or functor, a lambda's parameter or a range-`for` variable no longer links to a function or method elsewhere that only shares its name, like protocolbuffers/protobuf's code generators calling `format(…)` on their `Formatter& format` parameter, which linked to the conformance suite's `TestResult::format`, or facebook/rocksdb's tests calling a local `auto key = [](int i) {…};`. C++ finds the local first, so that is what the call calls. -- In TypeScript and JavaScript, an import of a package your project declares itself with `declare module '…'` in a `.d.ts` file, like a `typings/mammoth/index.d.ts` written for an untyped package or VS Code's own `vscode.d.ts`, now links to that declaration, so `codegraph callers`, impact and `codegraph affected` show the code that uses your typings instead of nothing or a project symbol that only shares a name. Named, default and namespace imports, `export =` and renaming export lists are read the way TypeScript reads them, a function such a file declares without a body (`export function convertToHtml(…): Promise;`) is now indexed so calls to it have something to link to, and only an import of its package reaches it. A `declare module` that augments a package's own types, a wildcard like `declare module '*.svg'`, and copies of typings kept in test fixtures are left alone. Re-index TypeScript projects after upgrading. +- In TypeScript and JavaScript, an import of a package your project declares itself with `declare module '…'` in a `.d.ts` file, like a `typings/mammoth/index.d.ts` written for an untyped package or VS Code's own `vscode.d.ts`, now links to that declaration, so `codegraph callers`, impact and `codegraph affected` show the code that uses your typings instead of nothing or a project symbol that only shares a name. Named, default and namespace imports, `export =` and renaming export lists are read the way TypeScript reads them, a function such a file declares without a body (`export function convertToHtml(…): Promise;`) is now indexed so calls to it have something to link to, and only an import of its package reaches it. In a monorepo such a declaration counts in the package that holds it, the packages inside that one and the packages that depend on it, and an import of one of your own workspace packages links to that package's real module even where a `.d.ts` stubs it for TypeScript. A `declare module` that augments a package's own types, a wildcard like `declare module '*.svg'`, and copies of typings kept in test fixtures are left alone. Re-index TypeScript projects after upgrading. - In TypeScript, an interface that extends other interfaces, like `interface UserRepo extends Repo, Named`, is now linked to each of them: the type hierarchy shows them above the interface and above every class that implements it, a change to an interface reaches the interfaces built on it and their implementations in impact results, and a call on a value typed by such an interface, like `hass.formatEntityName(…)` on a `HomeAssistant` that extends seven interfaces, links to the method the interface inherits instead of a same-named method elsewhere. Before, only a class's `extends` and `implements` were read, so every interface stood alone. A type from a package, like `ButtonProps` from `@mui/material`, or one of TypeScript's own, like `Omit<…>`, still links to nothing rather than to a type of the same name elsewhere in your project. Re-index TypeScript projects after upgrading. - In TypeScript and JavaScript, a type, class, constant or other declaration that a `.d.ts` file declares inside `declare module '…'` now links only from files that use that module: files that import it, the module's other declarations, and files that import it through a file of yours that re-exports it. Before, any file using the same name linked to it: Node's `import { readFileSync } from 'fs'` linked to the `fs` that VS Code's `vscode.d.ts` declares, and a `ProcessEnv`, or an LSP `TextDocument` imported through a barrel, to the `vscode` module's types of those names. Re-index TypeScript projects after upgrading. - In C++, a call written through a class the index doesn't hold, like protocolbuffers/protobuf's `FeatureSet::descriptor()` or `OneofOptions::default_instance()` (both classes are declared in `descriptor.pb.h`, a file too large to index), no longer links to another class's method of that name whose class only shares a word with it, like `JavaFeatures_NestInFileClassFeature::descriptor` or `CppFileOptions::default_instance`. A class named like the call's would have been found first, so one that only shares a word is another class. A call through a template parameter, through an alias, including one the calling function or a macro it expands declares, or through a namespace the file opens, aliases or uses links as before, and so does a call of a method only one class in the project declares. Re-index C++ projects after upgrading. diff --git a/__tests__/ambient-module-imports.test.ts b/__tests__/ambient-module-imports.test.ts index abc125766..aa6e7a992 100644 --- a/__tests__/ambient-module-imports.test.ts +++ b/__tests__/ambient-module-imports.test.ts @@ -28,6 +28,10 @@ * shorthand `declare module 'x';` declares nothing to bind. Ambient function * signatures are extracted by both the native kernel (when built) and the wasm * extractor, which must agree. + * + * A declaration counts where the checker can see it: its own package, one + * nested under that, one depending on that. And an import of a module of the + * repository's own packages is that module, whatever `.d.ts` stubs its name. */ import { describe, it, expect, beforeAll, beforeEach, afterEach, afterAll } from 'vitest'; import * as fs from 'fs'; @@ -662,6 +666,318 @@ declare module 'typescript-lib' { }, 60_000); }); +describe('a declare module answers the package it is in', () => { + /** React's shape: the ESLint plugin stubs a Flow file of the `shared` package for tsc alone. */ + const FLAGS: Files = { + 'package.json': JSON.stringify({ name: 'root', private: true, workspaces: ['packages/*'] }), + 'packages/eslint/package.json': JSON.stringify({ name: 'eslint-plugin' }), + 'packages/eslint/src/shared/ReactFeatureFlags.d.ts': `declare module 'shared/ReactFeatureFlags' { + export const enableFoo: boolean; + export const onlyDeclared: boolean; +} +`, + 'packages/eslint/src/rule.ts': "import { enableFoo, onlyDeclared } from 'shared/ReactFeatureFlags';\nexport const on = enableFoo && onlyDeclared;\n", + 'packages/reconciler/package.json': JSON.stringify({ name: 'reconciler' }), + 'packages/reconciler/src/work.js': `import { enableFoo, flagName, onlyDeclared } from 'shared/ReactFeatureFlags'; +export function work() { + return enableFoo && onlyDeclared ? flagName() : null; +} +`, + 'packages/shared/package.json': JSON.stringify({ name: 'shared' }), + 'packages/shared/ReactFeatureFlags.js': "export const enableFoo = true;\nexport function flagName() { return 'foo'; }\n", + }; + + it('never takes a real module of the repository from another package', async () => { + const root = tempRoot(); + write(root, FLAGS); + const cg = await CodeGraph.init(root, { index: true }); + try { + const edges = edgesFrom(cg, 'packages/reconciler/src/work.js'); + expect(edges).toEqual(expect.arrayContaining([ + 'imports packages/shared/ReactFeatureFlags.js:enableFoo', + 'imports packages/shared/ReactFeatureFlags.js:flagName', + 'calls packages/shared/ReactFeatureFlags.js:flagName', + ])); + // Nothing, not even a name only the stub declares, crosses into the ESLint package. + expect(edges.filter((e) => e.includes('packages/eslint/'))).toEqual([]); + } finally { + cg.close(); + } + }, 60_000); + + it('gives way to the real module in its own package too, and answers what only it declares', async () => { + const root = tempRoot(); + write(root, FLAGS); + const cg = await CodeGraph.init(root, { index: true }); + try { + const edges = edgesFrom(cg, 'packages/eslint/src/rule.ts'); + expect(edges).toContain('imports packages/shared/ReactFeatureFlags.js:enableFoo'); + expect(edges).not.toContain('imports packages/eslint/src/shared/ReactFeatureFlags.d.ts:enableFoo'); + expect(edges).toContain('imports packages/eslint/src/shared/ReactFeatureFlags.d.ts:onlyDeclared'); + } finally { + cg.close(); + } + }, 60_000); + + it('keeps TypeScript’s order against a package whose module is only a declaration file', async () => { + const root = tempRoot(); + write(root, { + 'package.json': JSON.stringify({ name: 'root', private: true, workspaces: ['packages/*'] }), + 'packages/types-only/package.json': JSON.stringify({ name: 'types-only', types: 'index.d.ts' }), + 'packages/types-only/index.d.ts': 'export function make(): string;\n', + 'packages/app/package.json': JSON.stringify({ name: 'app' }), + 'packages/app/typings/own.d.ts': "declare module 'types-only' {\n export function make(): number;\n}\n", + 'packages/app/src/use.ts': "import { make } from 'types-only';\nexport const made = make();\n", + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + expect(edgesFrom(cg, 'packages/app/src/use.ts')).toEqual([ + 'calls packages/app/typings/own.d.ts:make', + 'imports packages/app/typings/own.d.ts:make', + 'imports packages/app/typings/own.d.ts:packages/app/typings/own.d.ts', + ]); + } finally { + cg.close(); + } + }, 60_000); + + it('links a module imported whole to the real file, and a namespace import’s members', async () => { + const root = tempRoot(); + write(root, { + ...FLAGS, + 'packages/eslint/src/whole.ts': "import * as flags from 'shared/ReactFeatureFlags';\nimport 'shared/ReactFeatureFlags';\nexport const name = flags.flagName();\n", + 'packages/reconciler/src/whole.js': "import * as flags from 'shared/ReactFeatureFlags';\nexport const name = flags.flagName();\n", + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + for (const file of ['packages/eslint/src/whole.ts', 'packages/reconciler/src/whole.js']) { + const edges = edgesFrom(cg, file); + expect(edges, file).toEqual(expect.arrayContaining([ + 'imports packages/shared/ReactFeatureFlags.js:packages/shared/ReactFeatureFlags.js', + 'calls packages/shared/ReactFeatureFlags.js:flagName', + ])); + expect(edges.filter((e) => e.includes('.d.ts')), file).toEqual([]); + } + } finally { + cg.close(); + } + }, 60_000); + + it('keeps a test’s stub to the tests of its own package, behind the real module', async () => { + // `packages/react/src/__tests__/testDefinitions/` declares `react`, the + // workspace's own package, and `prop-types`, an npm one, for a TypeScript test. + const root = tempRoot(); + write(root, { + 'package.json': JSON.stringify({ name: 'root', private: true, workspaces: ['packages/*'] }), + 'packages/react/package.json': JSON.stringify({ name: 'react', main: 'index.js' }), + 'packages/react/index.js': "export function createElement() { return null; }\n", + 'packages/react/src/__tests__/testDefinitions/React.d.ts': "declare module 'react' {\n export function createElement(): unknown;\n}\n", + 'packages/react/src/__tests__/testDefinitions/PropTypes.d.ts': "declare module 'prop-types' {\n export function string(): unknown;\n}\n", + 'packages/react/src/__tests__/Own-test.ts': "import { createElement } from 'react';\nimport { string } from 'prop-types';\nexport const made = [createElement(), string()];\n", + 'packages/dom/package.json': JSON.stringify({ name: 'dom', peerDependencies: { react: '*' } }), + 'packages/dom/src/__tests__/Dom-test.js': "import { createElement } from 'react';\nimport { string } from 'prop-types';\nexport const made = [createElement(), string()];\n", + 'packages/dom/src/render.js': "import { string } from 'prop-types';\nexport const made = string();\n", + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + const own = edgesFrom(cg, 'packages/react/src/__tests__/Own-test.ts'); + expect(own).toContain('calls packages/react/index.js:createElement'); + expect(own).toContain('calls packages/react/src/__tests__/testDefinitions/PropTypes.d.ts:string'); + expect(own.filter((e) => e.includes('React.d.ts'))).toEqual([]); + const other = edgesFrom(cg, 'packages/dom/src/__tests__/Dom-test.js'); + expect(other).toContain('calls packages/react/index.js:createElement'); + expect(other.filter((e) => e.includes('testDefinitions'))).toEqual([]); + expect(edgesFrom(cg, 'packages/dom/src/render.js')).toEqual([]); + } finally { + cg.close(); + } + }, 60_000); + + it('is no rival to a real namesake where it does not count', async () => { + // `createRef` passes through `packages/react`'s barrels as a binding the + // import resolver doesn't follow; a name match finds it, and a stub's + // bodiless `createRef` in another package must not stand in its way. + const root = tempRoot(); + write(root, { + 'package.json': JSON.stringify({ name: 'root', private: true, workspaces: ['packages/*'] }), + 'packages/react/package.json': JSON.stringify({ name: 'react', main: 'index.js' }), + 'packages/react/index.js': "export { createRef } from './src/ReactClient';\n", + 'packages/react/src/ReactClient.js': "import { createRef } from './ReactCreateRef';\nexport { createRef };\n", + // A `@flow` file is read as TSX, which ranks with the stub's TypeScript. + 'packages/react/src/ReactCreateRef.js': '/** @flow */\nexport function createRef() { return { current: null }; }\n', + 'packages/react/src/__tests__/testDefinitions/React.d.ts': "declare module 'react' {\n export function createRef(): unknown;\n}\n", + 'fixtures/clock/package.json': JSON.stringify({ name: 'clock', dependencies: { react: '^19.0.0' } }), + 'fixtures/clock/src/Clock.js': "import { createRef } from 'react';\nexport const faceRef = createRef();\n", + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + const edges = edgesFrom(cg, 'fixtures/clock/src/Clock.js'); + expect(edges).toContain('calls packages/react/src/ReactCreateRef.js:createRef'); + expect(edges.filter((e) => e.includes('.d.ts'))).toEqual([]); + } finally { + cg.close(); + } + }, 60_000); + + it('answers its own package when it is the only answer, and no sibling', async () => { + const stub = "declare module 'untyped-lib' {\n export function make(): string;\n export interface Made { id: string }\n}\n"; + const use = "import { make, Made } from 'untyped-lib';\nexport const made: Made = make();\n"; + const root = tempRoot(); + write(root, { + 'package.json': JSON.stringify({ name: 'mono', private: true, workspaces: ['packages/*'] }), + 'packages/typed/package.json': JSON.stringify({ name: 'typed', dependencies: { 'untyped-lib': '^1.0.0' } }), + 'packages/typed/typings/lib.d.ts': stub, + 'packages/typed/src/use.ts': use, + 'packages/typed/src/deep/nested/use.ts': use, + 'packages/listed/package.json': JSON.stringify({ name: 'listed', dependencies: { 'untyped-lib': '^1.0.0' } }), + 'packages/listed/src/use.ts': use, + 'packages/unlisted/package.json': JSON.stringify({ name: 'unlisted' }), + 'packages/unlisted/src/use.ts': use, + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + for (const file of ['packages/typed/src/use.ts', 'packages/typed/src/deep/nested/use.ts']) { + expect(edgesFrom(cg, file), file).toEqual([ + 'calls packages/typed/typings/lib.d.ts:make', + 'imports packages/typed/typings/lib.d.ts:Made', + 'imports packages/typed/typings/lib.d.ts:make', + 'imports packages/typed/typings/lib.d.ts:packages/typed/typings/lib.d.ts', + 'references packages/typed/typings/lib.d.ts:Made', + ]); + } + expect(edgesFrom(cg, 'packages/listed/src/use.ts')).toEqual([]); + expect(edgesFrom(cg, 'packages/unlisted/src/use.ts')).toEqual([]); + } finally { + cg.close(); + } + }, 60_000); + + it('answers a package that depends on the one it is in', async () => { + // Astro's `client.d.ts` declares `astro:assets`, and each project on `astro` reads it. + const use = "import { getImage } from 'astro:assets';\nexport const image = getImage();\n"; + const root = tempRoot(); + write(root, { + 'package.json': JSON.stringify({ name: 'astro-monorepo', private: true, workspaces: ['packages/*', 'examples/*'] }), + 'packages/astro/package.json': JSON.stringify({ name: 'astro' }), + 'packages/astro/client.d.ts': "declare module 'astro:assets' {\n export function getImage(): string;\n}\n", + 'packages/astro/src/own.ts': use, + 'packages/mdx/package.json': JSON.stringify({ name: '@astrojs/mdx', peerDependencies: { astro: '^7.0.0' } }), + 'packages/mdx/src/server.ts': use, + 'examples/blog/package.json': JSON.stringify({ name: '@example/blog', dependencies: { astro: '^7.0.0' } }), + 'examples/blog/src/pages/deep/index.ts': use, + 'examples/plain/package.json': JSON.stringify({ name: '@example/plain', dependencies: { other: '^1.0.0' } }), + 'examples/plain/src/index.ts': use, + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + for (const file of ['packages/astro/src/own.ts', 'packages/mdx/src/server.ts', 'examples/blog/src/pages/deep/index.ts']) { + expect(edgesFrom(cg, file), file).toContain('calls packages/astro/client.d.ts:getImage'); + } + expect(edgesFrom(cg, 'examples/plain/src/index.ts')).toEqual([]); + } finally { + cg.close(); + } + }, 60_000); + + it('answers the packages nested under the one it is in', async () => { + // VS Code's `src/vscode-dts/vscode.d.ts` is the root package's; each extension + // is a package of its own that its tsconfig points at the file. + const root = tempRoot(); + write(root, { + 'package.json': JSON.stringify({ name: 'code-oss' }), + 'src/vscode-dts/vscode.d.ts': "declare module 'vscode' {\n export class Disposable { dispose(): void; }\n export namespace window { export const active: string; }\n}\n", + 'extensions/git/package.json': JSON.stringify({ name: 'git' }), + 'extensions/git/src/main.ts': "import { Disposable } from 'vscode';\nexport const held = new Disposable();\n", + 'extensions/git/test/fixtures/app/package.json': JSON.stringify({ name: 'fixture-app' }), + 'extensions/git/test/fixtures/app/use.ts': "import { Disposable } from 'vscode';\nexport const held = new Disposable();\n", + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + for (const file of ['extensions/git/src/main.ts', 'extensions/git/test/fixtures/app/use.ts']) { + expect(edgesFrom(cg, file), file).toEqual(expect.arrayContaining([ + 'imports src/vscode-dts/vscode.d.ts:Disposable', + 'instantiates src/vscode-dts/vscode.d.ts:Disposable', + ])); + } + } finally { + cg.close(); + } + }, 60_000); + + it('takes a marker package.json for no package', async () => { + // `{"type":"commonjs"}` says how Node reads a folder, as vscode's `scripts/` and `test/` do. + const root = tempRoot(); + write(root, { + 'package.json': JSON.stringify({ name: 'app', dependencies: { 'untyped-lib': '^1.0.0' } }), + 'src/esm/package.json': JSON.stringify({ type: 'module' }), + 'src/esm/lib.d.ts': "declare module 'untyped-lib' {\n export function make(): string;\n}\n", + 'src/use.ts': "import { make } from 'untyped-lib';\nexport const made = make();\n", + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + expect(edgesFrom(cg, 'src/use.ts')).toContain('calls src/esm/lib.d.ts:make'); + } finally { + cg.close(); + } + }, 60_000); + + it('takes the nearest declaration a package can see, past a sibling’s nearer one', async () => { + const root = tempRoot(); + write(root, { + 'package.json': JSON.stringify({ name: 'mono', private: true, workspaces: ['apps/*'] }), + 'typings/lib.d.ts': "declare module 'untyped-lib' {\n export function make(): string;\n}\n", + 'apps/a/package.json': JSON.stringify({ name: 'a' }), + 'apps/a/src/deep/lib.d.ts': "declare module 'untyped-lib' {\n export function make(): number;\n}\n", + 'apps/a/src/deep/use.ts': "import { make } from 'untyped-lib';\nexport const made = make();\n", + 'apps/b/package.json': JSON.stringify({ name: 'b' }), + 'apps/b/src/deep/use.ts': "import { make } from 'untyped-lib';\nexport const made = make();\n", + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + expect(edgesFrom(cg, 'apps/a/src/deep/use.ts')).toContain('calls apps/a/src/deep/lib.d.ts:make'); + expect(edgesFrom(cg, 'apps/b/src/deep/use.ts')).toContain('calls typings/lib.d.ts:make'); + } finally { + cg.close(); + } + }, 60_000); + + it('merges a block only with the blocks its package can see', async () => { + const root = tempRoot(); + write(root, { + 'package.json': JSON.stringify({ name: 'mono', private: true, workspaces: ['packages/*'] }), + 'packages/a/package.json': JSON.stringify({ name: 'a' }), + // A class is a candidate of the fuzzy match too, which the gate on its answer stops. + 'packages/a/lib.d.ts': "declare module 'untyped-lib' {\n export interface Made extends Base {}\n export class Built extends Core {}\n}\n", + 'packages/b/package.json': JSON.stringify({ name: 'b' }), + 'packages/b/lib.d.ts': "declare module 'untyped-lib' {\n export interface Base { id: string }\n export class Core {}\n}\n", + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + expect(edgesFrom(cg, 'packages/a/lib.d.ts')).toEqual([]); + } finally { + cg.close(); + } + }, 60_000); + + it('still takes a package’s declaration through a file that reaches it', async () => { + // Importing the declaring file brings its block into the program, wherever it lives. + const root = tempRoot(); + write(root, { + 'package.json': JSON.stringify({ name: 'mono', private: true, workspaces: ['packages/*'] }), + 'packages/a/package.json': JSON.stringify({ name: 'a' }), + 'packages/a/lib.d.ts': "declare module 'untyped-lib' {\n export interface Made { id: string }\n}\n", + 'packages/b/package.json': JSON.stringify({ name: 'b' }), + 'packages/b/src/use.ts': "import type { Made } from '../../a/lib';\nexport function id(made: Made): string { return made.id; }\n", + }); + const cg = await CodeGraph.init(root, { index: true }); + try { + expect(edgesFrom(cg, 'packages/b/src/use.ts')).toContain('references packages/a/lib.d.ts:Made'); + } finally { + cg.close(); + } + }, 60_000); +}); + /** Every edge, by its ends' natural keys, with where it is written and how it resolved. */ function edgesOf(cg: CodeGraph): string[] { const keys = new Map( @@ -748,4 +1064,83 @@ export function run(options: MammothOptions): void { cg.close(); } }, 120_000); + + describe('beside a module of the repository', () => { + const packages: Files = { + 'package.json': JSON.stringify({ name: 'root', private: true, workspaces: ['packages/*'] }), + 'packages/eslint/package.json': JSON.stringify({ name: 'eslint-plugin' }), + 'packages/eslint/src/flags.d.ts': "declare module 'shared/flags' {\n export const enableFoo: boolean;\n}\n", + 'packages/eslint/src/rule.ts': "import { enableFoo } from 'shared/flags';\nexport const on = enableFoo;\n", + 'packages/reconciler/package.json': JSON.stringify({ name: 'reconciler' }), + 'packages/reconciler/src/work.js': "import { enableFoo } from 'shared/flags';\nexport const on = enableFoo;\n", + 'packages/shared/package.json': JSON.stringify({ name: 'shared' }), + 'packages/shared/other.js': 'export const other = 1;\n', + }; + const real: Files = { 'packages/shared/flags.js': 'export const enableFoo = true;\n' }; + + it('moves its package’s importers to a real module that appears', async () => { + const root = tempRoot(); + write(root, packages); + const cg = await CodeGraph.init(root, { index: true }); + try { + expect(edgesFrom(cg, 'packages/eslint/src/rule.ts')).toContain('imports packages/eslint/src/flags.d.ts:enableFoo'); + expect(edgesFrom(cg, 'packages/reconciler/src/work.js')).toEqual([]); + write(root, real); + await cg.sync(); + expect(edgesFrom(cg, 'packages/eslint/src/rule.ts')).toContain('imports packages/shared/flags.js:enableFoo'); + expect(edgesFrom(cg, 'packages/reconciler/src/work.js')).toContain('imports packages/shared/flags.js:enableFoo'); + expect(edgesOf(cg)).toEqual(await fullIndexOf({ ...packages, ...real })); + } finally { + cg.close(); + } + }, 120_000); + + it('moves its package’s importers to a package that appears', async () => { + const root = tempRoot(); + const { 'packages/shared/package.json': manifest, 'packages/shared/other.js': other, ...withoutPackage } = packages; + write(root, withoutPackage); + const cg = await CodeGraph.init(root, { index: true }); + try { + expect(edgesFrom(cg, 'packages/eslint/src/rule.ts')).toContain('imports packages/eslint/src/flags.d.ts:enableFoo'); + write(root, { 'packages/shared/package.json': manifest!, 'packages/shared/other.js': other!, ...real }); + await cg.sync(); + expect(edgesFrom(cg, 'packages/eslint/src/rule.ts')).toContain('imports packages/shared/flags.js:enableFoo'); + expect(edgesOf(cg)).toEqual(await fullIndexOf({ ...packages, ...real })); + } finally { + cg.close(); + } + }, 120_000); + + it('takes its package’s importers back when the real module goes', async () => { + const root = tempRoot(); + write(root, { ...packages, ...real }); + const cg = await CodeGraph.init(root, { index: true }); + try { + expect(edgesFrom(cg, 'packages/eslint/src/rule.ts')).toContain('imports packages/shared/flags.js:enableFoo'); + fs.rmSync(path.join(root, 'packages/shared/flags.js')); + await cg.sync(); + expect(edgesFrom(cg, 'packages/eslint/src/rule.ts')).toContain('imports packages/eslint/src/flags.d.ts:enableFoo'); + expect(edgesFrom(cg, 'packages/reconciler/src/work.js')).toEqual([]); + expect(edgesOf(cg)).toEqual(await fullIndexOf(packages)); + } finally { + cg.close(); + } + }, 120_000); + + it('leaves the real module’s importers on it when a stub is added', async () => { + const root = tempRoot(); + const { 'packages/eslint/src/flags.d.ts': stub, ...withoutStub } = packages; + write(root, { ...withoutStub, ...real }); + const cg = await CodeGraph.init(root, { index: true }); + try { + write(root, { 'packages/eslint/src/flags.d.ts': stub! }); + await cg.sync(); + expect(edgesFrom(cg, 'packages/eslint/src/rule.ts')).toContain('imports packages/shared/flags.js:enableFoo'); + expect(edgesFrom(cg, 'packages/reconciler/src/work.js')).toContain('imports packages/shared/flags.js:enableFoo'); + expect(edgesOf(cg)).toEqual(await fullIndexOf({ ...packages, ...real })); + } finally { + cg.close(); + } + }, 120_000); + }); }); diff --git a/src/db/queries.ts b/src/db/queries.ts index e470ab87a..acfe3fd7a 100644 --- a/src/db/queries.ts +++ b/src/db/queries.ts @@ -3838,6 +3838,31 @@ export class QueryBuilder { })); } + /** + * The module edges (`imports` edges into a file node) into declaration + * files, with the source file and language a resurrection needs: where an + * import of a package lands while a `declare module` of it is all there is. + */ + getModuleImportsIntoDeclarationFiles(): Array { + const rows = this.db + .prepare( + `SELECT e.*, tgt.file_path AS target_file_path, src.file_path AS source_file_path, src.language AS source_language + FROM nodes tgt + JOIN edges e ON e.target = tgt.id AND e.kind = 'imports' + JOIN nodes src ON src.id = e.source + WHERE tgt.kind = 'file' + AND (tgt.file_path LIKE '%.d.ts' OR tgt.file_path LIKE '%.d.mts' OR tgt.file_path LIKE '%.d.cts')` + ) + .all() as Array; + return rows.map((row) => ({ + ...rowToEdge(row), + edgeId: row.id, + targetFilePath: row.target_file_path, + sourceFilePath: row.source_file_path, + sourceLanguage: row.source_language, + })); + } + /** * The module edges (`imports` edges into a file node) into the files an * import of `dir/name` can find instead of a file added there: `dir/name.*` diff --git a/src/resolution/ambient-modules.ts b/src/resolution/ambient-modules.ts index 17f5a052c..a503bebb0 100644 --- a/src/resolution/ambient-modules.ts +++ b/src/resolution/ambient-modules.ts @@ -17,6 +17,15 @@ * position inside it. A wildcard pattern (`declare module '*.svg'`) applies * only once a specifier fails to resolve to a file, and a shorthand `declare * module 'x';` declares nothing to bind; no import is bound to either. + * + * A block is in force only where the checker sees it, and each package of a + * monorepo has a program of its own, so it answers its own package and the ones + * that read its typings, not a sibling's (see reachesProgram). And it only + * describes a module this repository implements itself: an import of a + * workspace package's module is that module (see isRepositoryModule in + * ./import-resolver), which is what runs. React's ESLint plugin stubs + * `shared/ReactFeatureFlags` for tsc; the reconciler's import of that name, and + * its own, are `packages/shared`'s. */ import type { Node } from '../types'; import type { ResolutionContext, UnresolvedRef } from './types'; @@ -592,6 +601,8 @@ interface Memo { merged: Map; /** Each read file's namespace blocks (see namespaceBlocks). */ namespaces: Map; + /** The package directly in each folder asked about (see packageManifest). */ + manifests: Map; } const MEMO = new WeakMap(); @@ -603,7 +614,7 @@ export function clearAmbientModuleMemos(context: ResolutionContext): void { function memoFor(context: ResolutionContext): Memo { let memo = MEMO.get(context); - if (!memo) MEMO.set(context, (memo = { scanned: null, members: new Map(), declarationIds: null, otherBlocks: new Map(), declarations: new Map(), merged: new Map(), namespaces: new Map() })); + if (!memo) MEMO.set(context, (memo = { scanned: null, members: new Map(), declarationIds: null, otherBlocks: new Map(), declarations: new Map(), merged: new Map(), namespaces: new Map(), manifests: new Map() })); return memo; } @@ -692,12 +703,76 @@ function blockMembers(mod: AmbientModule, context: ResolutionContext): Map isTestFile(file); +const dirOf = (file: string): string => file.slice(0, Math.max(0, file.lastIndexOf('/'))); + +/** What a package.json says of its package. */ +interface PackageManifest { + name: string | null; + dependencies: Set; +} + +const DEPENDENCY_FIELDS = ['dependencies', 'devDependencies', 'peerDependencies', 'optionalDependencies']; + +/** The package whose package.json is `text`; null for none, as for a `{"type":"module"}`, which says how Node reads a folder. */ +function parseManifest(text: string | null): PackageManifest | null { + if (!text) return null; + try { + const json = JSON.parse(text) as Record | null; + if (!json || typeof json !== 'object') return null; + const name = typeof json.name === 'string' && json.name ? json.name : null; + const dependencies = new Set(DEPENDENCY_FIELDS.flatMap((field) => Object.keys((json[field] ?? {}) as object))); + return name !== null || dependencies.size > 0 ? { name, dependencies } : null; + } catch { + return null; + } +} + +/** The package whose package.json is directly in the folder `dir`, if any. */ +function packageManifest(dir: string, context: ResolutionContext): PackageManifest | null { + const memo = memoFor(context).manifests; + let manifest = memo.get(dir); + if (manifest === undefined) { + manifest = parseManifest(context.readFile(dir ? `${dir}/package.json` : 'package.json')); + memo.set(dir, manifest); + } + return manifest; +} + +/** The folder of the package `dir` is in: the nearest at or above it with a package of its own, '' for the repository's. */ +function packageFolder(dir: string, context: ResolutionContext): string { + return dir === '' || packageManifest(dir, context) ? dir : packageFolder(dirOf(dir), context); +} + /** - * The script declarations of `specifier` an import in `fromFile` can mean, - * the files nearest it first. One a test or a fixture keeps counts only for a - * test's imports: copies of vscode's API typings under copilot's - * `test/fixtures` are left out of its build and shadow nothing. + * Whether a `declare module` in `file` can be in the program `fromFile` is in. + * Each package of a monorepo has a program of its own, so it can if `fromFile` + * is in the package `file` is in, or in a package nested under that one: + * vscode's extensions are packages of their own, and each has its tsconfig read + * `src/vscode-dts/vscode.d.ts`, the root package's. It can too if `fromFile`'s + * package depends on `file`'s, whose typings it reads: `import … from + * 'astro:assets'` in an Astro project, a dependent of `astro`, is the module + * `astro/client.d.ts` declares. No other package's: react's + * `packages/eslint-plugin-react-hooks` stubs `shared/ReactFeatureFlags` for itself. + * + * One a test or a fixture keeps counts only for a test's imports, and only in + * its own package, as no dependent ever reads it: copies of vscode's API typings + * under copilot's `test/fixtures` are left out of its build and shadow nothing. */ +function reachesProgram(file: string, test: boolean, fromFile: string, fromTest: boolean, context: ResolutionContext): boolean { + if (test && !fromTest) return false; + const folder = packageFolder(dirOf(file), context); + if (folder === '' || fromFile.startsWith(`${folder}/`)) return true; + if (test) return false; + const name = packageManifest(folder, context)?.name; + return !!name && !!packageManifest(packageFolder(dirOf(fromFile), context), context)?.dependencies.has(name); +} + +/** Whether a `declare module` in `declaringFile` counts for a file in `fromFile` (see reachesProgram). */ +export function declaringFileAnswers(declaringFile: string, fromFile: string, context: ResolutionContext): boolean { + return reachesProgram(declaringFile, isTestCode(declaringFile), fromFile, isTestCode(fromFile), context); +} + +/** The script declarations of `specifier` an import in `fromFile` can mean (see reachesProgram), the files nearest it first. */ function declarationsFor(specifier: string, fromFile: string, context: ResolutionContext): AmbientModule[] { const all = scanned(context).bySpecifier.get(specifier); if (!all) return []; @@ -707,7 +782,7 @@ function declarationsFor(specifier: string, fromFile: string, context: Resolutio if (cached) return cached; const fromTest = isTestCode(fromFile); const dir = fromFile.slice(0, fromFile.lastIndexOf('/') + 1); - const usable = all.filter((m) => m.script && (fromTest || !m.test)); + const usable = all.filter((m) => m.script && reachesProgram(m.file, m.test, fromFile, fromTest, context)); const from = dir.split('/'); const shared = (file: string): number => { const parts = file.split('/'); diff --git a/src/resolution/import-resolver.ts b/src/resolution/import-resolver.ts index 0bb67f9e9..64196938b 100644 --- a/src/resolution/import-resolver.ts +++ b/src/resolution/import-resolver.ts @@ -15,7 +15,7 @@ import { resolveWorkspaceImport, workspaceImportTargets, type WorkspaceImportTar import { stripCommentsForRegex, templateLiteralTextAt } from './strip-comments'; import { makeLineAt } from './synth-utils'; import { dartDirectiveFile } from './dart-libraries'; -import { ambientModuleDeclares, findAmbientModuleExport, findAmbientModuleFile, inBlock, isInModuleBlock, namespaceBlocks, type NamespaceBlock } from './ambient-modules'; +import { ambientModuleDeclares, declaringFileAnswers, findAmbientModuleExport, findAmbientModuleFile, inBlock, isDeclarationFile, isInModuleBlock, namespaceBlocks, type NamespaceBlock } from './ambient-modules'; import { isTestFile } from '../search/query-utils'; import { moduleReferenceKeys, moduleTail } from '../db/reference-tail'; import { @@ -2839,12 +2839,8 @@ function isJsBareModuleRef(ref: UnresolvedRef, context: ResolutionContext): bool */ export function jsModuleFile(ref: UnresolvedRef, context: ResolutionContext): string | null { if (!isJsPathImportRef(ref) && !isJsBareModuleRef(ref, context)) return null; - // A package the repository declares itself is the file that declares it, - // whatever file its specifier resolves to (see ./ambient-modules). - if (isBareModuleSpecifier(ref.referenceName, ref.language)) { - const declaring = findAmbientModuleFile(ref.referenceName, ref.filePath, context); - if (declaring) return declaring.filePath; - } + const declaring = declaredModuleFile(ref.referenceName, ref, context); + if (declaring) return declaring.filePath; const file = resolveImportPath(ref.referenceName, ref.filePath, ref.language, context); return file && file !== ref.filePath ? file : null; } @@ -2991,19 +2987,16 @@ function typeQualifier(ref: UnresolvedRef, context: ResolutionContext): string[] */ function importedTypeMember(imp: ImportMapping, path: string[], ref: UnresolvedRef, context: ResolutionContext): Node | null | undefined { const file = resolveImportPath(imp.source, ref.filePath, ref.language, context); - const bare = isBareModuleSpecifier(imp.source, ref.language); let found: Node | undefined; let rest: string[]; if (imp.isNamespace) { const want = { isDefault: false, isNamespace: true, exportedName: path[0]!, memberName: path[0]! }; - found = (bare ? findAmbientModuleExport(imp.source, want, ref, context) : undefined) ?? - (file ? findExportedSymbol(file, want, ref.language, context, new Set()) : undefined); + found = findImportedExport(imp.source, file, want, ref, context); if (found && path.length === 1) return found; rest = path.slice(1); } else { const want = { isDefault: imp.isDefault, isNamespace: false, exportedName: imp.isDefault ? 'default' : imp.exportedName, memberName: null }; - found = (bare ? findAmbientModuleExport(imp.source, want, ref, context) : undefined) ?? - (file ? findExportedSymbol(file, want, ref.language, context, new Set()) : undefined); + found = findImportedExport(imp.source, file, want, ref, context); rest = path; } if (found) { @@ -3382,20 +3375,15 @@ export function resolveViaImport( context ); - // A package the repository declares itself, as cord-field's - // `typings/mammoth/index.d.ts` declares `declare module 'mammoth'`: - // TypeScript takes that declaration before any file the specifier - // resolves to (see ./ambient-modules). - const bare = isBareModuleSpecifier(imp.source, ref.language); - if (resolvedPath || bare) { + // A package the repository declares itself may resolve to no file at all. + if (resolvedPath || isBareModuleSpecifier(imp.source, ref.language)) { const exportedName = imp.isDefault ? 'default' : imp.exportedName; const memberName = imp.isNamespace ? ref.referenceName.replace(imp.localName + '.', '') : null; const want = { isDefault: imp.isDefault, isNamespace: imp.isNamespace, exportedName, memberName }; - const targetNode = (bare ? findAmbientModuleExport(imp.source, want, ref, context) : undefined) ?? - (resolvedPath ? findExportedSymbol(resolvedPath, want, ref.language, context, new Set()) : undefined); + const targetNode = findImportedExport(imp.source, resolvedPath, want, ref, context); if (targetNode) { // `Foo.bar()` / `Foo.CONST` — a NAMED (non-namespace) class import @@ -3489,6 +3477,55 @@ function isBareModuleSpecifier(source: string, language: Language): boolean { return ESM_IMPORT_LANGUAGES.has(language) && !source.startsWith('.') && !source.startsWith('/'); } +/** + * Whether `specifier`, resolving to `file` from `ref`'s file, names a module of + * a package this repository holds: a workspace member or a path dependency. + * That module is implemented here, and a `declare module` of its name only + * describes it to the type checker: react's ESLint plugin stubs + * `shared/ReactFeatureFlags` because tsc can't read that Flow file, and what + * runs is the file. A file a `paths` alias names is another matter: the checker + * takes the declared module first and never reads it, as copilot's shim of + * `vscode` for its test runner. + */ +function isRepositoryModule(specifier: string, file: string | null, ref: UnresolvedRef, context: ResolutionContext): boolean { + if (file === null || isDeclarationFile(file)) return false; + const workspaces = context.getWorkspacePackages?.(); + return !!workspaces && workspaceImportTargets(specifier, workspaces, ref.filePath) !== null; +} + +/** + * What an import of `want` from `source` names, `file` being where `source` + * resolves to, if anywhere: the export of a module of the repository's own + * packages, then the repository's own `declare module` of the package, as + * cord-field's `typings/mammoth/index.d.ts` declares `mammoth`, then the export + * of any other file, which TypeScript takes the declaration before (see + * ./ambient-modules). + */ +function findImportedExport( + source: string, + file: string | null, + want: { isDefault: boolean; isNamespace: boolean; exportedName: string; memberName: string | null }, + ref: UnresolvedRef, + context: ResolutionContext +): Node | undefined { + const inFile = (): Node | undefined => (file ? findExportedSymbol(file, want, ref.language, context, new Set()) : undefined); + const declared = isBareModuleSpecifier(source, ref.language) ? findAmbientModuleExport(source, want, ref, context) : undefined; + if (!declared) return inFile(); + return (isRepositoryModule(source, file, ref, context) ? inFile() : undefined) ?? declared; +} + +/** + * The file that declares `specifier` as a module, for `ref`'s file (see + * ./ambient-modules): the module, whatever file the specifier resolves to, + * short of a module of the repository's own packages. + */ +function declaredModuleFile(specifier: string, ref: UnresolvedRef, context: ResolutionContext): Node | undefined { + if (!isBareModuleSpecifier(specifier, ref.language)) return undefined; + const declaring = findAmbientModuleFile(specifier, ref.filePath, context); + if (!declaring) return undefined; + return isRepositoryModule(specifier, resolveImportPath(specifier, ref.filePath, ref.language, context), ref, context) ? undefined : declaring; +} + /** * Reference kinds the extractor records by a qualified name's last part: a * type position (`doc: vscode.TextDocument`) and a construction (`new @@ -3514,13 +3551,16 @@ const LAST_NAME_KINDS: ReadonlySet = new Set(['references', 'extends', ' * A name bound to another package (an LSP `TextDocument`), to Node's own * modules (`fs`), or to a project file that doesn't import the module, or a * bare name a file reaching no such module writes, is not one of the module's. + * Neither is one the module's name alone leads to, in a block that doesn't + * count for the file (see declaringFileAnswers). */ export function reachesAmbientModule(ref: UnresolvedRef, specifier: string, declaringFile: string, context: ResolutionContext): boolean { - if (isInModuleBlock(ref.filePath, ref.line, ref.column ?? 0, specifier, context)) return true; + const answers = declaringFileAnswers(declaringFile, ref.filePath, context); + if (answers && isInModuleBlock(ref.filePath, ref.line, ref.column ?? 0, specifier, context)) return true; const mappings = context.getImportMappings(ref.filePath, ref.language); const relative = (source: string): boolean => source.startsWith('.') || source.startsWith('/'); const leadsTo = (source: string, through: (file: string) => boolean): boolean => { - if (source === specifier) return true; + if (source === specifier) return answers; if (!relative(source)) return false; const file = resolveImportPath(source, ref.filePath, ref.language, context); return file !== null && (file === declaringFile || through(file)); @@ -3850,12 +3890,8 @@ function resolveModuleImportToFile( continue; } - // A package the repository declares itself in a script `.d.ts`: that file - // is the module, whatever file the specifier resolves to (see ./ambient-modules). - if (isBareModuleSpecifier(modulePath, ref.language)) { - const declaring = findAmbientModuleFile(modulePath, ref.filePath, context); - if (declaring) return { original: ref, targetNodeId: declaring.id, confidence: 0.9, resolvedBy: 'import' }; - } + const declaring = declaredModuleFile(modulePath, ref, context); + if (declaring) return { original: ref, targetNodeId: declaring.id, confidence: 0.9, resolvedBy: 'import' }; const resolvedPath = resolveImportPath(modulePath, ref.filePath, ref.language, context); if (resolvedPath && resolvedPath !== ref.filePath) { diff --git a/src/resolution/module-importers.ts b/src/resolution/module-importers.ts index 757a19fd1..2f2315017 100644 --- a/src/resolution/module-importers.ts +++ b/src/resolution/module-importers.ts @@ -477,7 +477,9 @@ interface OutrankedImport { * file of the package an added file is in: the entry its manifest declares * (`./src/index.ts`, or the source of `./dist/index.js`), or the source of a * path into its build output, outranks the file an import fell back to while - * it was missing, such as the folder's own index (#2502). + * it was missing, such as the folder's own index (#2502). And the imports into + * a declaration file: a module of the project's own package outranks a + * `declare module` of its name (see ./ambient-modules). */ function outrankedImports( queries: QueryBuilder, @@ -487,6 +489,12 @@ function outrankedImports( ): OutrankedImport[] { const out: OutrankedImport[] = []; const packageImports = new Map>(); + let intoDeclarations: ReturnType | undefined; + type ModuleImport = ReturnType[number]; + const byName = (edge: ModuleImport): boolean => { + const specifier = edge.metadata?.refName; + return typeof specifier === 'string' && !specifier.startsWith('.') && !specifier.startsWith('/'); + }; for (const file of added) { if (!BINDING_LANGUAGES.has(file.language)) continue; const { dir, name } = importedName(file.path); @@ -495,14 +503,16 @@ function outrankedImports( if (packageDir) { let imports = packageImports.get(packageDir); if (!imports) { - imports = queries.getModuleImportsUnder(packageDir).filter((edge) => { - const specifier = edge.metadata?.refName; - return typeof specifier === 'string' && !specifier.startsWith('.') && !specifier.startsWith('/'); - }); + imports = queries.getModuleImportsUnder(packageDir).filter(byName); packageImports.set(packageDir, imports); } + intoDeclarations ??= queries.getModuleImportsIntoDeclarationFiles().filter(byName); const seen = new Set(edges.map((edge) => edge.edgeId)); - for (const edge of imports) if (!seen.has(edge.edgeId)) edges.push(edge); + for (const edge of [...imports, ...intoDeclarations]) { + if (seen.has(edge.edgeId)) continue; + seen.add(edge.edgeId); + edges.push(edge); + } } for (const edge of edges) { const specifier = edge.metadata?.refName; diff --git a/src/resolution/name-matcher.ts b/src/resolution/name-matcher.ts index e9bf7b27d..35349d5dd 100644 --- a/src/resolution/name-matcher.ts +++ b/src/resolution/name-matcher.ts @@ -23,7 +23,7 @@ import { cppClassWritten, type CppNameSite } from './cpp-supertypes'; import { cppIncludedFile, cppIncludedHeadersBeside, cppIncluders } from './cpp-includers'; import { isCFeatureVariant, isOffByDefault } from './c-feature-variants'; import { isTestFile, isTestPath } from '../search/query-utils'; -import { inBlock, isDeclarationFile, isMergedAmbientDeclaration, scanGlobalScope, type GlobalScopeScan } from './ambient-modules'; +import { ambientModuleDeclarations, declaringFileAnswers, inBlock, isDeclarationFile, isMergedAmbientDeclaration, scanGlobalScope, type GlobalScopeScan } from './ambient-modules'; import { isMinifiedContent } from '../extraction/generated-detection'; import { getCargoWorkspaceCrateMap, parsePackageName, parseTargetPaths } from './frameworks/cargo-workspace'; import { blankRustCode, readRustScopes, rustBindingAt, rustInnerBlockAt, rustBoundsOf, rustFnAt, rustImplAt, splitRustTopLevel, type RustBinding, type RustPatternSite, type RustScopes } from './rust-scopes'; @@ -13717,6 +13717,7 @@ export function matchByExactName( // Every candidate uses the same binding key. Keep this check lazy. let locallyBoundJsName: boolean | undefined; const tsBareTypeName = (): boolean => (bareTypeName ??= isTsBareTypeName(ref, context)); + const ambientDeclarations = ambientModuleDeclarations(context); const sameName = outOfRepoCall ? innerScopeShadows(context.getNodesByName(ref.referenceName), ref, context) : context.getNodesByName(ref.referenceName); @@ -13826,6 +13827,10 @@ export function matchByExactName( // its own or a global, never another module's — ahead of the ranking, so // a global `interface Window` in a `declare global` still wins. !(n.filePath !== ref.filePath && tsBareTypeName() && !isTsTypeInScope(n, ref, context)) && + // Nor is a `declare module` block a file can't see any candidate, ahead of + // the ranking, where the gate on the winner would drop the reference: a + // sibling's test stub's `createRef` is no rival to `packages/react`'s own. + !(ambientDeclarations.has(n.id) && !declaringFileAnswers(n.filePath, ref.filePath, context)) && // Likewise for `imports`: a member that only exists inside a type is not // importable, so it is not a candidate. Without this a `path`/`id`/`url` // import resolved to some interface's same-named property.