Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,16 @@ This repo keeps test fixtures out of default Codegraph scans with `codegraph.con
{
"discovery": {
"ignoreGlobs": ["tests/samples/**", "tests/languages/samples/**"]
},
"languages": {
"extensions": {
".tpl": "php"
}
}
}
```

Use this pattern in other repos when large fixture, generated, or vendored trees should not participate in search, unresolved-import checks, graphing, indexing, inspect, impact, or review runs. Config globs are project-root-relative. CLI `--include-glob` and `--ignore-glob` stay relative to each active scan root.
Use discovery globs in other repos when large fixture, generated, or vendored trees should not participate in search, unresolved-import checks, graphing, indexing, inspect, impact, or review runs. Config globs are project-root-relative, while CLI `--include-glob` and `--ignore-glob` stay relative to each active scan root. `languages.extensions` maps additional or built-in file suffixes to supported language IDs; longer suffixes win, and built-ins remain active unless explicitly remapped.

## Quick start

Expand Down
3 changes: 2 additions & 1 deletion codegraph-skill/codegraph/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ Fall back to CLI when MCP is unavailable.

## Discovery

Durable repo-local ignores belong in `codegraph.config.json`.
Durable repo-local ignores and custom language suffixes belong in `codegraph.config.json`.
Use `languages.extensions` for suffix-to-language mappings such as `.tpl` to `php`; keys start with `.`, values are supported language IDs, and the longest suffix wins.
One-off CLI filters use scan-root-relative `--include-glob` and `--ignore-glob`.
Use `--no-gitignore` only when ignored files are intentionally in scope.

Expand Down
11 changes: 10 additions & 1 deletion docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,18 +36,27 @@ Commands that scan a project read `codegraph.config.json` from `--root` when it
"includeGlobs": ["src/**/*.ts"],
"ignoreGlobs": ["tests/samples/**", "tests/languages/samples/**"],
"useGitignore": true
},
"languages": {
"extensions": {
".tpl": "php",
".inc.php": "php",
".build.ts": "ts"
}
}
}
```

- `discovery.includeGlobs` and `discovery.ignoreGlobs` are project-root-relative, even when a command scans child include roots.
- `discovery.ignoreGlobs` is for large fixture, generated, or vendored folders that should not be indexed.
- `languages.extensions` maps additional or built-in suffixes to supported language IDs; keys must start with `.`, values must name a supported language, and the longest suffix wins.
- Built-in suffixes remain active unless explicitly remapped by `languages.extensions`.
- CLI `--include-glob` and `--ignore-glob` values are one-off additions relative to each scanned root.
- `inspect` follow-up commands preserve the selected `--root` and include roots.
- `--no-gitignore` overrides `useGitignore`.

Config globs and one-off CLI globs apply at different layers. `codegraph.config.json` globs are durable and project-root-relative. CLI scan-root globs are additive for a single command and are evaluated relative to each active scan root. `--no-gitignore` disables `.gitignore` filtering for that command only; it does not change config.
Cache and manifest reuse is rooted at `--root`. Reusing a project root lets commands share compatible index and graph entries when the file signatures, config, graph options, and relevant build options still match. Changing `--root`, changing discovery config, or changing graph options creates a different reuse boundary. Child include-root scans can reuse project-root cache entries, but command summaries and follow-up commands stay scoped to the selected include roots.
Configured language extensions automatically extend discovery for matching files and participate in cache compatibility checks. Reusing a project root lets commands share compatible index and graph entries when the file signatures, config, graph options, and relevant build options still match. Changing `--root`, changing discovery or language-extension config, or changing graph options creates a different reuse boundary. Child include-root scans can reuse project-root cache entries, but command summaries and follow-up commands stay scoped to the selected include roots.

## Core commands

Expand Down
8 changes: 8 additions & 0 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,7 @@ async function runCliWithActiveRuntime(rawArgs: string[]) {
return {
...(progressHandler ? { onProgress: progressHandler } : {}),
discovery: discoveryOptions,
...(config.languages?.extensions ? { languageExtensions: config.languages.extensions } : {}),
...(cache !== undefined ? { cache } : {}),
...(hasFlag("--cache-strict") ? { cacheStrict: true } : {}),
...(hasFlag("--cache-verify") ? { cacheVerify: true } : {}),
Expand Down Expand Up @@ -756,6 +757,7 @@ async function runCliWithActiveRuntime(rawArgs: string[]) {
discoveryOptions,
nativeMode,
workerOpts,
languageExtensions: config.languages?.extensions,
progressHandler,
graphOptions: hasGraphOverrides ? buildGraphOptions() : undefined,
reportEnabled,
Expand Down Expand Up @@ -785,6 +787,7 @@ async function runCliWithActiveRuntime(rawArgs: string[]) {
onProgress: progressHandler,
discovery: discoveryOptions,
...(nativeMode !== "auto" ? { native: nativeMode } : {}),
...(config.languages?.extensions ? { languageExtensions: config.languages.extensions } : {}),
...workerOpts,
},
writeJSONLine,
Expand All @@ -807,6 +810,7 @@ async function runCliWithActiveRuntime(rawArgs: string[]) {
discovery: discoveryOptions,
...(hasGraphOverrides ? { graph: buildGraphOptions() } : {}),
...(nativeMode !== "auto" ? { native: nativeMode } : {}),
...(config.languages?.extensions ? { languageExtensions: config.languages.extensions } : {}),
...workerOpts,
},
writeJSONLine,
Expand Down Expand Up @@ -956,6 +960,7 @@ async function runCliWithActiveRuntime(rawArgs: string[]) {
discovery: discoveryOptions,
...(graphOptions ? { graph: graphOptions } : {}),
...(nativeMode !== "auto" ? { native: nativeMode } : {}),
...(config.languages?.extensions ? { languageExtensions: config.languages.extensions } : {}),
...workerOpts,
});

Expand Down Expand Up @@ -1008,6 +1013,7 @@ async function runCliWithActiveRuntime(rawArgs: string[]) {
projectRootFs,
includeRootsAbs,
discoveryOptions,
languageExtensions: config.languages?.extensions,
graphOptions: hasGraphOverrides || nativeMode !== "auto" ? buildGraphOptions() : undefined,
nativeMode,
workerOpts,
Expand All @@ -1027,6 +1033,7 @@ async function runCliWithActiveRuntime(rawArgs: string[]) {
projectRootFs,
includeRootsAbs,
discoveryOptions,
languageExtensions: config.languages?.extensions,
graphOptions: hasGraphOverrides || nativeMode !== "auto" ? buildGraphOptions() : undefined,
nativeMode,
workerOpts,
Expand Down Expand Up @@ -1058,6 +1065,7 @@ async function runCliWithActiveRuntime(rawArgs: string[]) {
onProgress: progressHandler,
discovery: discoveryOptions,
...(nativeMode !== "auto" ? { native: nativeMode } : {}),
...(config.languages?.extensions ? { languageExtensions: config.languages.extensions } : {}),
...workerOpts,
},
});
Expand Down
2 changes: 2 additions & 0 deletions src/cli/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ export type IndexCommandContext = {
discoveryOptions: ProjectFileDiscoveryOptions;
nativeMode: NativeRuntimeMode;
workerOpts: { useNativeWorkers: true } | Record<string, never>;
languageExtensions: BuildOptions["languageExtensions"];
progressHandler: ((update: { current: number; total: number }) => void) | undefined;
graphOptions: GraphBuildOptions | undefined;
reportEnabled: boolean;
Expand Down Expand Up @@ -66,6 +67,7 @@ export async function handleIndexCommand(context: IndexCommandContext): Promise<
threads,
discovery: context.discoveryOptions,
...(context.nativeMode !== "auto" ? { native: context.nativeMode } : {}),
...(context.languageExtensions ? { languageExtensions: context.languageExtensions } : {}),
...context.workerOpts,
...(cache !== undefined ? { cache } : {}),
cacheStrict,
Expand Down
29 changes: 21 additions & 8 deletions src/cli/inspect.ts
Original file line number Diff line number Diff line change
@@ -1,23 +1,25 @@
import fs from "node:fs";
import path from "node:path";
import { findDuplicates, type DuplicateConfidence, type DuplicateGroup } from "../duplicates.js";
import { findDuplicates } from "../duplicates.js";
import type { DuplicateConfidence, DuplicateGroup } from "../duplicates.js";
import { collectGraph } from "../graph-builder.js";
import { findDetailedCycles, getUnresolvedImports } from "../graphs/queries.js";
import { getHotspots } from "../graphs/hotspots.js";
import { type GraphBuildOptions } from "../graphs/types.js";
import type { GraphBuildOptions } from "../graphs/types.js";
import { buildProjectIndexIncremental } from "../indexer/build-index.js";
import { type BuildReport } from "../indexer/types.js";
import type { BuildReport } from "../indexer/types.js";
import {
getNativeTreeSitterLoadError,
getNativeTreeSitterSupportedLanguageIds,
isNativeTreeSitterAvailable,
type NativeRuntimeMode,
} from "../native/treeSitterNative.js";
import type { NativeRuntimeMode } from "../native/treeSitterNative.js";
import type { Graph } from "../types.js";
import { restrictGraphToIncludeRoots } from "../util/includeRoots.js";
import { supportForFile } from "../languages.js";
import type { LanguageExtensionMap } from "../languages.js";
import { toProjectDisplayPath } from "../util/paths.js";
import { type ProjectFileDiscoveryOptions } from "../util/projectFiles.js";
import type { ProjectFileDiscoveryOptions } from "../util/projectFiles.js";
import { parseCacheModeOption, parsePositiveIntegerOption } from "./options.js";

type CacheMode = "off" | "memory" | "disk";
Expand Down Expand Up @@ -94,6 +96,7 @@ export type InspectCommandContext = {
projectRootFs: string;
includeRootsAbs: string[];
discoveryOptions: ProjectFileDiscoveryOptions;
languageExtensions: LanguageExtensionMap | undefined;
graphOptions: GraphBuildOptions | undefined;
nativeMode: NativeRuntimeMode;
workerOpts: { useNativeWorkers: true } | Record<string, never>;
Expand Down Expand Up @@ -149,6 +152,7 @@ async function buildScopedReportGraph(
opts: {
cache?: CacheMode;
discovery?: ProjectFileDiscoveryOptions;
languageExtensions?: LanguageExtensionMap;
graphOptions?: GraphBuildOptions;
nativeMode?: NativeRuntimeMode;
workerOpts?: { useNativeWorkers: true } | Record<string, never>;
Expand All @@ -165,6 +169,7 @@ async function buildScopedReportGraph(
files,
cache: "disk",
...(opts.discovery ? { discovery: opts.discovery } : {}),
...(opts.languageExtensions ? { languageExtensions: opts.languageExtensions } : {}),
...(opts.progressHandler ? { onProgress: opts.progressHandler } : {}),
...(opts.nativeMode && opts.nativeMode !== "auto" ? { native: opts.nativeMode } : {}),
...(opts.workerOpts ?? {}),
Expand All @@ -179,6 +184,7 @@ async function buildScopedReportGraph(

const sourceGraph = await collectGraph(projectRoot, files, {
...(opts.graphOptions ?? {}),
...(opts.languageExtensions ? { languageExtensions: opts.languageExtensions } : {}),
...(opts.report ? { report: opts.report } : {}),
});
return {
Expand Down Expand Up @@ -208,10 +214,13 @@ function summarizeDuplicateGroup(group: DuplicateGroup): DuplicateOpportunitySum
};
}

function countFilesByLanguage(files: Iterable<string>): Record<string, number> {
function countFilesByLanguage(
files: Iterable<string>,
languageExtensions?: LanguageExtensionMap,
): Record<string, number> {
const byLanguage: Record<string, number> = {};
for (const file of files) {
const languageId = supportForFile(file)?.id ?? "other";
const languageId = supportForFile(file, languageExtensions)?.id ?? "other";
byLanguage[languageId] = (byLanguage[languageId] ?? 0) + 1;
}
return byLanguage;
Expand Down Expand Up @@ -248,6 +257,7 @@ async function buildInspectReport(
files: string[],
discovery: ProjectFileDiscoveryOptions,
graphOptions: GraphBuildOptions | undefined,
languageExtensions: LanguageExtensionMap | undefined,
cache: CacheMode | undefined,
nativeMode: NativeRuntimeMode,
workerOpts: { useNativeWorkers: true } | Record<string, never>,
Expand All @@ -264,6 +274,7 @@ async function buildInspectReport(
files,
cache: cache ?? "disk",
discovery,
...(languageExtensions ? { languageExtensions } : {}),
...(progressHandler ? { onProgress: progressHandler } : {}),
...(nativeMode !== "auto" ? { native: nativeMode } : {}),
...workerOpts,
Expand Down Expand Up @@ -297,7 +308,7 @@ async function buildInspectReport(
},
files: {
total: files.length,
byLanguage: countFilesByLanguage(files),
byLanguage: countFilesByLanguage(files, languageExtensions),
},
hotspots,
unresolved: {
Expand Down Expand Up @@ -343,6 +354,7 @@ export async function handleInspectCommand(context: InspectCommandContext): Prom
files,
context.discoveryOptions,
context.graphOptions,
context.languageExtensions,
cache,
context.nativeMode,
context.workerOpts,
Expand All @@ -361,6 +373,7 @@ export async function handleHotspotsCommand(context: InspectCommandContext): Pro
const { graph } = await buildScopedReportGraph(context.projectRootFs, context.includeRootsAbs, files, {
...(cache ? { cache } : {}),
discovery: context.discoveryOptions,
...(context.languageExtensions ? { languageExtensions: context.languageExtensions } : {}),
...(context.graphOptions ? { graphOptions: context.graphOptions } : {}),
nativeMode: context.nativeMode,
workerOpts: context.workerOpts,
Expand Down
51 changes: 48 additions & 3 deletions src/config.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
import fsp from "node:fs/promises";
import path from "node:path";
import { z } from "zod";
import { type ProjectFileDiscoveryOptions } from "./util/projectFiles.js";
import { supportById } from "./languages.js";
import type { LanguageExtensionMap } from "./indexer/types.js";
import type { ProjectFileDiscoveryOptions } from "./util/projectFiles.js";

export const CODEGRAPH_CONFIG_FILE = "codegraph.config.json";

const stringArraySchema = z.array(z.string().trim().min(1));

const languageExtensionsSchema = z.record(z.string().trim().min(1), z.string().trim().min(1));

const codegraphConfigSchema = z
.object({
discovery: z
Expand All @@ -17,13 +21,22 @@ const codegraphConfigSchema = z
})
.strict()
.optional(),
languages: z
.object({
extensions: languageExtensionsSchema.optional(),
})
.strict()
.optional(),
})
.strict();

type ParsedCodegraphConfig = z.infer<typeof codegraphConfigSchema>;

export type CodegraphConfig = {
discovery?: ProjectFileDiscoveryOptions;
languages?: {
extensions?: LanguageExtensionMap;
};
};

function uniq(values: readonly string[]): string[] {
Expand Down Expand Up @@ -78,6 +91,33 @@ function normalizeDiscoveryConfig(
return hasDiscoveryOptions(normalized) ? normalized : undefined;
}

function normalizeLanguageExtensions(extensions: Record<string, string> | undefined): LanguageExtensionMap | undefined {
if (!extensions) return undefined;
const normalized: LanguageExtensionMap = {};
for (const [rawKey, rawLanguageId] of Object.entries(extensions)) {
const key = rawKey.trim().toLowerCase();
const languageId = rawLanguageId.trim();
if (!key.startsWith(".")) {
throw new Error(`Invalid ${CODEGRAPH_CONFIG_FILE}: languages.extensions key "${rawKey}" must start with ".".`);
}
if (!supportById(languageId)) {
throw new Error(
`Invalid ${CODEGRAPH_CONFIG_FILE}: languages.extensions["${rawKey}"] references unknown language "${languageId}".`,
);
}
normalized[key] = languageId;
}
return Object.keys(normalized).length ? normalized : undefined;
}

function languageExtensionIncludeGlobs(languageExtensions: LanguageExtensionMap | undefined): string[] {
return Object.keys(languageExtensions ?? {})
.map((extension) => extension.trim().toLowerCase())
.filter((extension) => extension.startsWith("."))
.sort()
.map((extension) => `**/*${extension}`);
}

export async function loadCodegraphConfig(projectRoot: string): Promise<CodegraphConfig> {
const configPath = path.join(projectRoot, CODEGRAPH_CONFIG_FILE);
let raw: string;
Expand All @@ -102,8 +142,13 @@ export async function loadCodegraphConfig(projectRoot: string): Promise<Codegrap
if (!parsed.success) {
throw new Error(`Invalid ${CODEGRAPH_CONFIG_FILE}: ${z.prettifyError(parsed.error)}`);
}
const discovery = normalizeDiscoveryConfig(parsed.data.discovery);
const languageExtensions = normalizeLanguageExtensions(parsed.data.languages?.extensions);
const languageDiscovery = languageExtensions
? { includeGlobs: languageExtensionIncludeGlobs(languageExtensions) }
: undefined;
const discovery = mergeDiscoveryOptions(normalizeDiscoveryConfig(parsed.data.discovery), languageDiscovery);
return {
...(discovery ? { discovery } : {}),
...(hasDiscoveryOptions(discovery) ? { discovery } : {}),
...(languageExtensions ? { languages: { extensions: languageExtensions } } : {}),
};
}
11 changes: 8 additions & 3 deletions src/graph-builder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,15 @@ import type { Edge, Graph } from "./types.js";
import { loadWorkspaceConfig } from "./util/workspace.js";
import { normalizeResolutionHints } from "./util/paths.js";
import { mapLimit } from "./util/concurrency.js";
import { logWithLevel, type LogLevel } from "./logging.js";
import { isNativeRequiredUnavailableError, type NativeRuntimeMode } from "./native/treeSitterNative.js";
import { logWithLevel } from "./logging.js";
import type { LogLevel } from "./logging.js";
import { isNativeRequiredUnavailableError } from "./native/treeSitterNative.js";
import type { NativeRuntimeMode } from "./native/treeSitterNative.js";
import { initNativeBackendReport } from "./native/nativeBackendReport.js";
import { collectAngularJsFrameworkEdges } from "./graphs/angularjs.js";
import { type FallbackImportExtractionEvent } from "./graphs/specifiers.js";
import type { FallbackImportExtractionEvent } from "./graphs/specifiers.js";
import type { GraphCacheEntry } from "./graphs/types.js";
import type { LanguageExtensionMap } from "./languages.js";
import type { BuildReport } from "./indexer/types.js";
import type { ParsedFileContext } from "./indexer/parse-context.js";
import { collectEdgesForFile } from "./graph-edge-collector.js";
Expand Down Expand Up @@ -57,6 +60,7 @@ export async function collectGraph(
replaceFiles?: Set<string>;
logLevel?: LogLevel;
allFiles?: string[];
languageExtensions?: LanguageExtensionMap;
},
): Promise<Graph> {
const normalizePath = (file: string) => file.replace(/\\/g, "/");
Expand Down Expand Up @@ -146,6 +150,7 @@ export async function collectGraph(
...(opts?.report ? { report: opts.report } : {}),
allFiles: normalizedAllFiles,
...(sqlFactCache ? { sqlFactCache } : {}),
...(opts?.languageExtensions ? { languageExtensions: opts.languageExtensions } : {}),
});
addEdgeTargetsToGraph(edges);
return edges;
Expand Down
Loading