Repository navigation
TypeScript test selection: leave out tests that never load the change, type tests, package-imported names (stacked on #1888) - #1893
Merged
Conversation
A formatter puts a long test name on a line of its own: `it(` or `test.each(rows)(` opens the call, the name follows, and the arrow starts a later line that names no registrar. The rule that names an anonymous callable a test reads only its own declaration line, so the runner's test was test code but no test: a file whose tests are all written this way was never selected, and the change it covers listed it only as "a route this count does not credit". The callable is now a test when its line starts with the function value and the innermost multi-line call opened above it names a registrar on its first line. Innermost, so a callback on its own line handed to `settle(` inside a test body stays that call's argument (control). The same helper serves the impact facts and the query layer's test sets. Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
A runner configuration names files it loads before every test file of the project (setupFiles, jest's setupFilesAfterEnv) and files whose exported setup it calls once before the run (globalSetup). Nothing imports or calls them, so a function they reach -- a client a set-up file configures at module level, a code generator a global set-up runs -- reached no test, while breaking it fails every test of the run. ax_runner_setup reads the string entries of those keys in vitest/vite/jest configuration files (relative, `<rootDir>/`, `__dirname + '/...'`, jest's rootDir), skipping a package's own set-up. The set-up file's module, and for a global set-up the function it exports to the runner (`export const setup = codegen`, `export default fn`, a declared setup/teardown), is a fixture (runs_before) of each test under the configuration's root; a global set-up's failure aborts the whole run, so its root is the nearest directory above that lists projects. Control: a helper file no configuration names reaches no test. Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
…the function object runs it
A callable API is often a function object merged with its members and typed by a callable interface whose call
signature has no body: `export const widget: Factory = assign(createWidget, members)`,
`Object.assign(task, { started, finished })`. The holder held the merge's result, for which nothing returned a
function, so a call of the merged object ran nothing and everything below the function had no caller from the
package's own tests or from the packages that import it.
value_branch now follows `Object.assign(target, ...sources)` to its first argument, written out or through an alias
whose value is `Object.assign` (`export const assign = Object.assign`), never by the bare name. Control: a function
passed as a later source is not what a call of the result runs.
Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
…d set's implementation The compiler binds `pick(1)` to `function pick(x: number): number`, a declaration with no body, and call_edges records exactly that. The implementation below the signatures -- the one body that runs -- had no caller in any verb: an edit to it reached no test (`tests`, `impact <file:line>`: "the change is local"), anything it calls was cut off from the tests above it, and `path` called the test and the implementation independent. The engine already answers which body runs (ext_call_runs_edge, from call_runs_method). graph_sql.overload_impl_edges pairs it with the call site that selected a same-named signature declared above the implementation (same file, same owner), and every reader takes the pair: impact's calls facts (a verified edge, at the site's line), the fast path's edge list, and path, as an `overload` hop. Controls: a same-named function in another module is no part of the set, and a placeholder name (`<arrow>`, shared by every anonymous arrow of a module) names no set at all -- without that guard member-assigned-function-names cited a caller at another arrow's call. Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
Measured against runtime traces on the Python oracle corpus, every false
transitive-reach row has a static edge the runtime never made (the scorer
accepts any runtime caller at any depth, so context-insensitivity cannot
produce one). Two of the edge kinds are wrong by construction and are
fixed here; reach precision 0.675 -> 0.724, reach recall -0.1pt (src) /
-0.2pt (test-side), test-file recall -0.1pt, callers and path unchanged.
1. A decoration string naming a PARAMETER of the decorated declaration is
no registration key (ax_registration.decoration_key_strings, shape 4),
nor is a string inside another call in the decoration (shape 5).
`@option("--params", "-p", "params") def main(params)`: the flags are
what a caller writes to reach the command; the parameter name is written
by every function that builds a dict with that key, and the by-key join
made each of them a caller of the command. The flags stay keys.
2. The "decorator by name" hop lands on what the decorator RETURNS, not on
the decorator's body. It used to run decorated -> decorator, making the
decorated function (and everything reaching it: a route's tests) a
caller of whatever the decorator calls. Calling the decorated name never
runs the decorator; it runs what the decorator returned:
- bare `@d`: d's return; written as a call `@d(...)`: the return of
what d returned (decorated_call);
- a wrapper (fn_returns, from the engine's method_returns_method): the
decorated declaration now stands for that wrapper;
- the function handed back unchanged (fn_returns_param, a registering
decorator): no hop at all; the decoration itself runs at import,
which the import walk already follows;
- unknown return: the old hop stays (a route the walk cannot judge).
IMPACT_VERSION 70 (new facts fn_returns, fn_returns_param, decorated_call).
Cases: decorator-by-name-runs-its-wrapper, option-parameter-name-is-no-key
(each fails on the base, with a control that must not change);
decorator-the-project-declares now expects the route it really takes
(`at import`) instead of the by-name hop.
Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
… its class `cls: type[X] | None = None` and `Optional[Type[X]]` are the default-None spelling of a class-object parameter: the body swaps None for a default class and calls `cls(...)`. Only a top-level `type[...]` annotation was read, so the call stayed `callee_is_parameter` and every constructor behind it lost that caller (a decorator factory building its command class in a closure is the common shape). param_class_object_ref now names the `type[...]` subscript an annotation is or one of its union operands is, and both the concrete (expr_type_class_object) and the bounded-TypeVar (param_class_object_bound) rules read it. The existing top-level rules are unchanged. Corpus: one-hop callers src 0.828 -> 0.829, path found 0.727 -> 0.729, test->target path 0.559 -> 0.562; reach and tests unchanged; engine suite and torture unchanged (agree 588 / missing 46 / extra 37); index time flat. Case: optional-class-object-parameter (fails on the base; the bare `type[Command]` control passes on both). Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
…ile is left out
impact's test walk is a union over every caller: a call through a function type fans out to every function
of that type, and a hub reaches whatever any of its callers reaches, so a test of one middleware was selected
for every other middleware. A runtime probe of which test files enter each function found that none of the
wrongly selected files on the TypeScript subjects ran the function; most never import its file.
- imports_fact reads a JS / TS import over several lines (a one-line reading missed every
`import {\n a,\n} from './x'`, so a test of url.ts did not import url.ts), side-effect, dynamic and
jest / vi mock imports, import.meta.glob, tsconfig `paths` (scoped to their config), `#` subpath imports,
`.vue` / `.svelte` scripts, the root package's own name, a workspace package whose entry names dist/
(its source), and the JSX runtime a .tsx file loads without writing it. It is read off the code view, so an
import written inside a string or a comment is none. `import type` loads nothing and is skipped.
- imports_unknown names the files whose imports name this repository's code and resolve to nothing.
- loads_change counts the runner's set-up files as loaders, holds a route whose chain stops short of the
change (a by-name hop) to the seeds' files, and says nothing rather than no where an unread import is in
the closure.
- A JS / TS test whose loads_change is False is left out of `tests` and counted under `unloaded_tests`
(AXIOMCODE_KEEP_UNLOADED=1 keeps them). A route that crosses a process (remote, spawns, by key) stays.
Python is unchanged: its process also imports what plugins and entry points name.
- A vitest type test (`*.test-d.ts`, `*.spec-d.ts`) runs none of its code: it is listed with the stubs, which
break on a signature change and never on a body change, and is not counted.
- An unresolved bare call of a name the file imports from a third-party package is the package's function:
it takes the library kind (a row apart, no seed), and its import binding is no value reference to a
project declaration.
Cases: a-test-that-never-imports-the-change, a-type-test-runs-no-code, a-package-import-is-not-a-project-function
(each fails on the base for the precision check and keeps its controls).
Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
…m, walk them as [asserted] (#1892) An answer that stops at a call the graph could not resolve now lists the site (file:line:col, the call as written, the engine's reason). axiomcode link <file:line[:col]> <target> records where it lands in axiomcode-links.tsv; the graph validates each link (a call there consistent with the target, the target declared, the line text unchanged) and applies it as a call edge of tier asserted, never resolved. Links only add edges, survive rebuilds, follow their line by its text within the callable, and drop (reported) when the line is edited. A link whose target declares a return type also resolves calls chained on the result and calls on a local assigned from it. Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
…ha load-time throw selects the whole run
impact
- imports_fact resolves a directory import (`require('..')`, `require('../')`, a folder with a
package.json main/exports), the repository's own package name, a side-effect `import './x'` and a
literal `import('./x')`; a package whose entry is a build output the checkout lacks names its
src/ (or lib/) entry. imports_dynamic records the files that load a module by a computed name.
- loads_change asks whether the test loads the asked declaration's FILE, not the end of the
shortest route (which can run through another test file's subclass or callback); it says nothing
where the target file has no importer, the test file imports nothing here, or the test's import
closure holds a computed-name loader.
- a JavaScript test whose route is in-process (sound, one of a set, dispatch, registered, defines,
fixture) and whose file never loads the changed file is kept apart as `not_loading_tests` and
named on a `not counted:` line, never selected: a plugin's prototype override is credited only to
the tests that load the plugin, and a callback one test file hands to a shared function no longer
pulls in every other caller's tests. Key joins, remote hops and spawns are unaffected.
- load_run (impact.dl): test-code declarations that reach the change through calls alone; a test
credited through its module body or describe() block that is one of them carries `at_load`.
path
- from a JavaScript test file that never loads the target's file: "no route: X never loads Y".
test-impact
- a selected JavaScript file no runner collects is replaced by the collected test files that
require it, or moved to not_collected when none does (unless a test file loads modules by a
computed name).
- a mocha run whose specs reach the change at load time (`at_load`) lists every spec that run
collects: mocha loads them in one process and a throw there fails all of them. jest/vitest are
per-file and unchanged.
- a changed function-constructor (`function Foo(opts) {` header edit is a TYPE to `changed`) is asked
again as the function when impact finds no type at that line.
Measured on 8 JavaScript subjects against cached mutation truth (no-library protocol), base ->
this: impact R 0.522 -> 0.542, P 0.238 -> 0.295; tests R 0.494 -> 0.559, P 0.264 -> 0.310; held-out
impact P 0.156 -> 0.478 at equal recall; no failing (test file, target) pair lost in impact, tests
or path on any subject.
Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
IMPACT_VERSION 70 carries both 69s (the library-callback receiver sites and the JavaScript import facts). Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
…'s result (#1892) A site is file:line:col (the callee's name); two calls of one name on a line are never chosen between. link --not rejects a by-name or one-of-a-set lead at a site (never an edge the engine resolved); the walks skip it, nothing is deleted. AXIOMCODE_LINKS_PREFER=1 skips a linked site's own guesses. Unknown sites carry ranked candidates from what the graph knows (the engine's set, values passed in, a computed name's prefix, values registered in the file, the name called). A link whose target has a declared, JSDoc or inferred return type resolves the calls chained on its result, on a local assigned from it, awaited, or on each element it yields. Front-door answers list confirmed places before leads. Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
…cts a lead; candidates ranked by the name called (#1892) A decorator factory's call and the decoration applying its result are one call at one column. Stress harness scenarios S1-S13 pass in all five languages where the subject can exhibit them. Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
swapnilpaliwal-sd
marked this pull request as ready for review
October 10, 2026 17:40
swapnilpaliwal-sd
requested review from
JaredHLZhang,
Whua689 and
suyashpaliwal26
as code owners
October 10, 2026 17:40
…o fix/js-impact-f1 # Conflicts: # plugins/axiomcode/skills/axiomcode/scripts/axiomcode-impact
# Conflicts: # plugins/axiomcode/skills/axiomcode/scripts/axiomcode-impact
# Conflicts: # plugins/axiomcode/skills/axiomcode/scripts/axiomcode-impact # plugins/axiomcode/skills/axiomcode/scripts/axiomcode-path
# Conflicts: # plugins/axiomcode/skills/axiomcode/scripts/axiomcode-impact
Closed
5 tasks done
Contributor
Author
|
Tracked in #1897 (0.1.9 batch); merge in the order listed there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1888 (its commits show here until it merges). Draft: measured, not for merge yet.
What it fixes
impact's test walk is a union over every caller. A call through a function type fans out to every function of that type, and a hub reaches whatever any of its callers reaches, so a test of one middleware was selected for every other middleware. A runtime probe (one instrumented suite run per subject, recording which test file entered each mutated function) showed that on every traced TypeScript subject ~99% of wrongly selected test files never entered the function — the loss is static over-reach, not tests that ran it and passed.
unloaded_testsin--json, aleft out:line in text;AXIOMCODE_KEEP_UNLOADED=1keeps them). Routes that cross a process (remote,spawns,by key) stay. Python is unchanged.imports_factnow reads an import whose{ … }list spans lines (a one-line reading missed them: a test ofurl.tsdid not "import"url.ts), side-effect / dynamic /jest|vi.mockimports,import.meta.glob, tsconfigpaths(scoped),#imports,.vue/.sveltescripts, the root package's own name, a workspace package whose entry namesdist/(its source), and the JSX runtime a.tsxfile loads implicitly. Read off the code view (strings and comments blanked).import typeis skipped.imports_unknown: a file whose import names repository code and resolves to nothing makesloads_changesay nothing instead of no.setupFiles,globalSetup) count as loaders; a route whose chain stops short of the change (by name) is held to the seeds' files.*.test-d.ts,*.spec-d.ts) run none of their code: listed with the stubs ([type test]), not counted.import { waitFor } from '@testing-library/…') is the package's function: library kind (row apart, no seed), and its import binding is no value reference to a project declaration.Effect (TypeScript mutation kit, 9 subjects, base = #1888 tip)
Truth = test files that newly fail when the function throws on entry. 0 truth files lost on any subject, either verb.
Per subject, impact precision: router library 0.134 → 0.247, toolkit monorepo 0.418 → 0.610, RPC monorepo 0.338 → 0.411, statechart monorepo 0.265 → 0.288 (recall 0.623 → 0.652: the import reading also finds
.vueroutes), date library tests verb 0.326 → 0.467; the rest unchanged.Runtime caller oracle (callers / reach ≤ 3 hops / path), 4 subjects
Per-test-file call-site tracer (the TS runtime oracle, keyed by the running test file; suites pass unchanged under it), scored like the Python harness (direct
calls itrows; reach precision over predicted functions that ran; 25 runtime 2-call chains for path). Base and fix are identical on these verbs (the change is test selection):Cost
Interleaved on one subject (same graph, base and fix scripts alternated): warm
impact --testsmedian 3.29 s → 3.38 s; first query (facts export) 26 s → 22 s (noise). Index time unchanged (parser and Soufflé rules untouched).imports_fact918 → 907 rows there (type-only imports dropped, multi-line ones added).Checks
a-test-that-never-imports-the-change,a-type-test-runs-no-code,a-package-import-is-not-a-project-function.tests/run.py: typescript 280/328 (base 269/317: the same 48 failures, all pre-existing here; +11 new checks pass), javascript 302/319 = base, python 321/330 = base, java 342/343 = base, csharp 222/241 = base (identical failure sets).Not fixed (what stands between this and F1 0.8)
by keyroute is kept as possibly cross-process: on the RPC monorepo it keeps 602 wrong files (P would be 0.527) with 0 right ones; telling an in-process key dispatch from a broker needs a design call.date.getMonth()→getMonth): 189 wrong rows vs 21 right (JS tests throughrequirenamespaces), left alone.