You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This introduces common "verbs" (build, test, format, lint, generate) that individual parts of the project can implement, and the shared infrastructure in misc/just/ behind them.
This supersedes #19978, which had accumulated too much history to review. Same content, recreated as a readable commit series on top of main, plus the fixes listed below.
# name a single test directory — goes straight to the suite that owns it
just test rust/ql/test/query-tests/diagnostics
# aim at a tree — every recipe found underneath runs
just format cpp # → bazel files under cpp/, then cpp/ql QL formatting
just generate .# → rust, then swift# several arguments, grouped by the recipe that serves them
just build rust java
just test java/ql/test java/ql/integration-tests
# flags reach the runner
just test go/ql/test + # + is short for --all-checks# repo-wide: runs what is cheap, names what it passed over
just test.# each verb has a short alias: t, b, f, l, g (and gen)
just t rust/ql/test/query-tests/diagnostics
just f cpp
Forwarding
A verb is spelled the same everywhere, but what it means is defined per language, next to the code it acts on. The forwarder finds the justfiles implementing a verb for each of its arguments, so there is no central list of who implements what.
Justfiles are looked for in both directions from an argument:
above it, where the recipe is passed the argument itself, as that says what to act on
below it, where the recipe is passed its own directory, as there the argument only said where to look
Every distinct recipe found this way runs. Recipes are compared by value, so one reached through import is recognised as the same job and runs once, while a cross-cutting recipe higher up composes with the more specific ones below instead of hiding them. Arguments are grouped by the recipe they resolve to, so just build rust java works.
Two things keep the search useful:
a justfile can set explicit_verbs to stay out of reach of a verb aimed at one of its parents. QL test suites use this: just test . no longer sweeps whole language suites, which take a long time. A verb that passes over such a directory says so and names it.
a root that forwards a verb has already spent the plain name on the forwarder, so it names its own implementation _root_<verb>. That is how the root formats bazel files, which belong to no single language, while just format cpp stays within cpp.
Running QL tests
by default the CLI is built from the internal repo; nothing is built when working in codeql standalone
--all-checks, abbreviated +, adds the extra checks CI runs, configured per language
--codeql=built skips the build step, consistent with the same pytest option
set lists (casey/just#1988, now fixed) replaces the whitespace-separated string blobs that the old version used to encode recipe arguments, along with the re-splitting in the Python helpers. That encoding could not carry an argument faithfully: one containing a space was silently split in two, and a value that was set but empty could not be written at all, which is why the Kotlin tests spelled CODEQL_EXTRACTOR_KOTLIN_DIAGNOSTIC_LIMIT with a literal space, to leave something for the split to find.
Bugs fixed along the way:
formatting no longer breaks on the many paths in this repository containing spaces
go language-tests-386 is restored
RAM_PER_THREAD and the nolang search path are passed through
Caveats
When one verb runs several recipes, any non-positional argument has to be understood by all of them. This works for options like --learn or --codeql, which the shared test definitions all accept.
Companion PR
The internal repo companion PR ports all languages to this infrastructure and updates CI accordingly.
These queries live next to the C++ QL tests they check, so that `codeql test
run --consistency-queries` can find them without an internal checkout.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`lint.py --format-only` gives the upcoming `just format` verb a way to
reformat without failing on pre-existing lint findings.
codegen.sh looked up its runfiles via `external/ql+`, which only resolves
in a main-repository layout. `../ql+` works from both, so codegen keeps
working when the repository is consumed as a bazel dependency.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Introduces a small set of verbs (`test`, `build`, `generate`, `format`,
`lint`) that work the same from anywhere in the tree, so that contributors do
not have to remember a different incantation per language. Running a verb from
the root forwards it to whichever justfile actually implements it for the given
paths; running it in a language directory uses that language's definition
directly.
Everything language-specific stays in the per-language justfiles added next;
this commit only provides the vocabulary they share:
- `misc/just/forward.just` and `forward_command.py` resolve a verb plus a set
of paths to the justfiles that implement it, grouping paths per justfile.
- `misc/just/lib.just` exposes `_codeql_test`, `_language_tests` and
`_integration_test` for the per-language justfiles to build on.
- `codeql_test_run.py` turns test flags into a `codeql test run` invocation,
resolving `RAM_PER_THREAD`/`CPUS` from arguments, environment, then platform
defaults.
- `misc/just/defs.just` holds the settings and generic helpers, including the
internal-checkout detection that lets the same justfiles work in both repos.
Arguments are passed around as just lists (`set lists`), so values containing
spaces survive intact rather than being re-split by the helpers.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Each language declares its own test flags, consistency queries and build steps,
so that `just test`, `just build` and friends do the right thing wherever they
are run from. The flag sets are transcribed from the internal CI definitions
they replace, so behaviour is unchanged.
`unified` gets the same treatment as the other languages, including the
consistency queries that were previously not run anywhere.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pointing a verb at a directory that merely contains implementations used to
fail, so `just test cpp` was an error and reaching a suite required either
naming it in full or hand-writing an aggregate recipe.
When nothing at or above an argument implements the verb, look below it
instead. Justfiles are enumerated with `git ls-files`, which is two orders of
magnitude faster than walking a checkout with build outputs in it, and probed
in parallel with `just --dump`. That dump also replaces the previous trick of
recognising a forwarder by a string in its stderr: a recipe that depends on
`_forward` does not implement the verb, whichever repository it lives in.
Upward search still wins, so naming a recipe after a verb now decides what
that verb means for the whole subtree. `unified` was doing exactly that and
would have hidden its own QL tests, so its bazel entry point goes back to
being called `extractor-tests`. Directories that only make sense when named
explicitly say so with `explicit_verbs`.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Running a whole language suite means building a CodeQL CLI and waiting a long
time, which is not something `just test <anything above it>` should decide to do
on the user's behalf. Integration tests and the Kotlin CI shards already opted
out for the same reason; the suites themselves are the bigger case.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolution used to stop at the first justfile found walking up, and only looked
downwards when that found nothing. That assumed a recipe named after a verb
covers everything beneath it, which is not how these are written: `rust` formats
Rust sources while `rust/ql` formats QL, so `just format rust` silently skipped
the QL files.
Both directions are now searched and every distinct recipe runs. Recipes reached
through `import` are the same job rather than a new one, so they are recognised
as already covered and run once.
A cross-cutting recipe placed high up therefore composes with the ones below it
rather than shadowing them, which is the point: formatting Bazel files
repository-wide should add to what each directory does with its own sources, not
replace it.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`format` pasted a `find` command substitution straight into the shell, which split the
result on whitespace. Hundreds of query files live under directories such as `Best
Practices`, so `just format cpp` handed the formatter a nonexistent `./src/Best` and
died. Arguments given on the command line were torn apart the same way, which defeats
the point of `set lists`.
Collecting the files in a helper rather than with `find` also avoids two portability
traps: `find` on Windows is an unrelated program, and the full file list is well past
the command line length limit there, so it has to be batched.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Directories opting out of discovery were only named when a verb found nothing at all.
When it did find something, `just test .` looked like it had covered the tree while
quietly leaving eleven test suites alone. Report them whenever they are passed over.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`language-tests-386` was dropped when the Go justfile was ported, leaving two generated
workflows invoking a recipe that no longer existed.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Naming the directories a verb passed over was reported the same way as a verb matching
nothing at all, on stderr and marked as an error. A run that did everything asked of it
then looked like a failure. Report it as part of the account of what ran instead, and
keep the error for the case where nothing matched.
List one directory per line, as a verb aimed at a repository root passes over dozens,
and name the invocation that failed, as by then a verb may have fanned out widely.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
bazel files are identified by name rather than extension, and the tree of checked-in
generated ones has to stay out of any sweep over them. Matching globs against the file
name covers both the old extensions and those names, and exclusions keep the generated
files out.
Absolute names are an option because a command run through `bazel run` starts in the
runfiles directory, where a relative name means nothing. Splitting on the last `--`
lets such a command carry one of its own.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A repository root forwards every verb, and a recipe written beside the import replaces
the imported one, so it had no name left to implement a verb under: adding `format` to
the root broke `just format cpp` outright. Some work belongs to no single directory
though, and the root is where it should live.
Such a justfile now spells its own implementation `_root_<verb>`, which the forwarder
looks for whenever the plain name turns out to be the forwarder's. Taking an argument,
it composes with what is found below rather than shadowing it, so a verb aimed at a
subdirectory still reaches only that subdirectory.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
These are spread across the whole tree rather than gathered under a language, so they
are the root's to format, and `_root_format` keeps a run aimed at a subdirectory to the
bazel files under it.
The buildifier bazel target cannot be driven directly: the wrapper it generates ignores
the paths given to it and always sweeps the workspace. Running the binary instead means
supplying the exclusion of the checked-in generated files ourselves, which buildifier
has no flag for. Being a dev dependency, the target only resolves in a build rooted
here; inside the internal repository its own buildifier target covers these files.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`codeql query format` will only name the files it rewrites if it also names every file
it leaves alone, so asking which files changed meant thousands of lines to find them
in, and bazel was similarly talkative about building the formatter it was about to run.
The file runner can now be told which lines of a command's output to hide, so the
formatter is asked for everything and the lines about untouched files are dropped. It
is a denylist rather than a pick of what to keep, so errors and anything unforeseen
still come through, and the command's exit code is passed on unchanged.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Building the internal distribution printed its whole log every time, so any command
that needed one first said several dozen lines about unzipping a JDK before saying the
one thing it was asked to say.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Guarding this to standalone checkouts made it useless, as that is not where the work
happens. It was guarded because the target here is a bazel dev dependency, unreachable
from a build rooted in the internal repository; but both repositories depend on the
buildifier binary, each as the root module of its own checkout, so asking for it
directly resolves either way.
What differs is which bazel to ask and from where. The internal workspace encloses this
one, and this one is itself a bazel module, so a nested invocation would take the
enclosing checkout for something it is not; the file runner can now be told which
directory to run from, which is also what its absolute file names were already for.
Rewritten files are now named, as the QL formatter does, leaving out the accounting
buildifier gives for those it did not rewrite.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Needing a CodeQL CLI is not a reason to hide a suite, as every QL recipe here needs one
and the rest stay discoverable. Being slow is the whole of it.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A file named `BUILD.<something>` that is not `BUILD.bazel` is not a bazel file: bazel
knows `BUILD` and `WORKSPACE` by name and the rest by extension, so `BUILD.windows.tpl`
and its kind are templates, holding placeholders that no formatter can parse. Matching
them failed every format whose scope contained one, which the internal repository has
and this one does not.
Formatting now also asks bazel from the root of the checkout the files belong to, rather
than from the enclosing one when there is one. The buildifier behind it is a dependency
of whichever checkout is the root, so which one asks decides which version formats, and
the files of a repository are best formatted by the version it pins and skipped by the
list of generated files it keeps. Building goes the other way, as a target there needs
the enclosing workspace to resolve at all, so the two no longer share a helper.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The exclusions are one justfile variable, and a root defining its own bazel formatting
has more than one directory its generators write to. Reading them the way the file name
patterns are already read costs nothing and saves spelling the option twice.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A command spanning a line break left the sentence for it to rewrap, so formatting the
directory always came back with a change, and the rewrap broke out of the list it was in.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Asking bazel from this repository's root was half of formatting its own files and not
another's: the paths came from the verb rather than from the root, so a checkout
enclosing this one had its files formatted here, by whichever buildifier version this
repository happens to pin. The two are not interchangeable, differing in the fixes they
apply, so files came out formatted by a version other than the one their own repository
would use on them.
Bounding the files to the root leaves each repository formatting what it owns, and a
verb spanning both is answered once by each, every root implementing the verb for
itself.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…it is on
A single argument is capped far below the whole command line, at 128KB against 2MB on
Linux, and a command that passes its arguments on through a shell arrives as one of
them. Sizing batches by the line alone let a large enough tree build one argument over
that cap, which fails as an `execv` error from whatever did the handing on, naming
neither this file nor the files it was given.
Also says how an exclusion is matched, as it is against the path the walk built rather
than the one on the command line, and the natural way to name a directory only matches
when the walk starts above it.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Hand-sorting every argument was justified while `--all-checks` was both a
flag and an assignment, which argparse cannot express. Naming the offer
separately removed that, and the loop was then keeping three silent
mistakes alive: `--codeql built` made `built` a test path, and a bare
`--codeql` or `--extra-check` was forwarded to `codeql test run` to fail
there instead of here.
argparse takes those three options; the shape test stays for the rest,
which belongs to `codeql test run` and is forwarded untouched. Errors are
routed back through `error` so they still arrive in a just banner.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ARGS is accepted but never appended to the Bazel invocation, so forwarded options such as just test misc/codegen --test_output=errors are silently discarded. The other Bazel-backed test recipes concatenate their variadic arguments; this recipe should do the same.
Do not treat an unreadable justfile as successfully skipped
misc/just/forward_command.py:149
A failed just --dump is only printed and then removed from parsed. If another recipe is found, forwarding runs it and returns success even though the unreadable justfile may implement the requested verb, so automation can silently skip part of the requested tree. Propagate probe failures to forward and make the overall command fail after reporting them.
Propagate Git discovery failures
misc/just/forward_command.py:162
Returning an empty list makes a failed git ls-files indistinguishable from a directory containing no justfiles. When an enclosing recipe is still found, the command can complete with status 0 while all implementations below the argument were never discovered. Preserve this failure and return a nonzero forwarding status instead of treating it as an empty result.
Group invocations by recipe identity, not justfile path
misc/just/forward_command.py:359
The PR promises that imported recipes are compared by value and run once, but aggregation here keys only by the justfile path after resolve has discarded the recipe value. The same imported recipe reached through different justfiles across arguments therefore runs twice—for example, python/justfile imports python/ql/justfile, so just format python python/ql creates separate invocations for one QL-formatting recipe. Preserve the recipe identity from resolution and use it to group arguments while retaining one invocation context.
Require the internal checkout before creating the long path
python/justfile:9
In a standalone checkout SEMMLE_CODE is the empty stub value, so this recipe constructs /ql/python/... and attempts to create directories at the filesystem root before _language_tests can report that an internal checkout is required. Add _require_semmle_code as this helper's dependency so standalone invocations fail safely before running the body.
The paragraph above tells a root that adding to a shared value is a change
to make on both sides, and then the diagnostic beside it claimed it could
not see such a change. It can: read as two values rather than as two names,
the pair shows the pattern that never arrived. The reason to say so is that
the two lines differ in both directions, since overriding usually means
adding, and only one of the two differences is the bug.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The dataclass existed to merge two parses by hand, which was only needed
because the offered checks were applied in a second pass. Parsing the
original line again with them appended reaches the same place, so the
merging goes, and with it the reason to have a class at all.
`LANGUAGE` joins the parser as the positional it always was, which also
retires the hand-written usage check: a missing one is now reported the
same way as every other bad argument. `--` is consumed rather than
forwarded as a result, so `codeql test run` stops receiving a stray
separator ahead of ours.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`_rule` reads as a policy in a file that has several, and the value is a
line of `#`. `JUST_CMD_RULE` keeps its name, being the documented way to
preset one.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`namepath` is what anyone hunting one would try first, and it is a module
path, so two top-level recipes read alike and it says nothing about where
either was written. Worth naming so the next reader stops there rather
than checking.
Also records that no repository would notice this failure: defining a
recipe of one's own only moves it further from the imported one, so the
shape that would start running twice is the one this test builds.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The first definition of a variable wins, so the optional internal file has
to come before the stub that stands in for it. Reversed, the stub wins even
where the real file exists and that checkout quietly behaves as if it were
external.
Worth a comment rather than trusting the reader's intuition, which says
last wins, and because this repository cannot catch the mistake: the file
is never present here, so both orders evaluate identically.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
These examples use the internal checkout's ql/<language> layout, while this repository has rust/... and cpp/... (as line 17 itself notes). They therefore fail when copied from this README in codeql; moreover, cpp/ql/test opts out of downward test, so just test cpp would not illustrate that behavior. Use public-repository paths and a discoverable subtree such as unified for the downward-test example.
An assignment only reaches this loop by matching `ENV_RE`, which requires a
name before the `=`, so the key can never come back empty and the branch
could not run.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`set lists` is what makes argument forwarding work and it did not exist
before 1.58, but nothing here said so. The error an older `just` gives is
clear and points at the line, yet names no version to move to, which is
the gap worth closing in prose rather than with a check.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Two trailing-whitespace lines and a long call. `misc/just` is outside the black
scope in `.pre-commit-config.yaml`, so nothing in CI would have said so.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`env_value` is gone and `parse_arguments` reads `sys.argv` itself, so every test
that called either of them stopped running. Precedence is now decided in `main`,
where an assignment and an inherited variable first meet, so the tests about
precedence go through `main` rather than through a function that no longer exists.
A later empty value now erases an earlier setting instead of being skipped. That is
only about what this script reads back: every assignment is still exported, so an
empty one reaches the child set and empty rather than absent. Both halves are
pinned, since the first is easy to mistake for the second.
Empty arguments are dropped before parsing rather than while sorting. Only the `--`
case tells the two placements apart, so that is the test worth having.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The entry point sat two thirds of the way up the file, so running it as a
script stopped there: the thirteen tests defined below it never ran, and
`bazel test` reported a clean twenty-six with nothing to say it was a subset.
Among the missing were the checks that the fixtures still match what `just`
really dumps, and the one pinning that a recipe reached under two spellings
runs once.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`forward_command` honours JUST_EXECUTABLE while these tests went to PATH, so
pinning a `just` would have had the fixtures agree with one binary and the
code run another. Resolving the module's own setting also makes the tests
reachable under `bazel test`, whose sandbox offers a fixed PATH that no
`just` is on, so they skipped there on every machine.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`--all-checks` enables the checks a justfile offers; a root can separately pass
checks unconditionally, which are not offers. Nothing distinguished the two, and
they are easy to conflate when reading a justfile.
The flag is injected on every language test run rather than typed, so it means
"enable whatever this root offers" and a root offering nothing enables nothing.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Only the first argument finds a justfile; the rest are handed to that one
`test` recipe. The docstring said the first "must be a test root", which
reads as though the others are roots that get visited in turn, and it has
now been read that way by someone porting a suite onto it.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Both entry points dropped empty arguments before looking at them, to absorb a
caller interpolating a variable that was never set. No justfile here can produce
one: a root that contributes nothing contributes no list element, so the case
the filtering existed for cannot arise, and dropping arguments silently is a bad
way to find out otherwise.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The prefix was written `{error}`, which is Python's spelling. just interpolates
with `{{ }}`, so every internal-checkout failure said `{error}` where it meant a
red `error:`. It is the only single-brace interpolation in any justfile here.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`misc/codegen`'s `test` declared arguments and passed none on, so anything given
to it was accepted and discarded.
`_ensure_long_filename` ran before the dependency that checks for an internal
checkout, so outside one it built a path from an unset variable and created
directories at the filesystem root, or failed on permissions while saying
nothing about the real reason.
The Kotlin shards pointed at a recipe name that exists nowhere. They are reached
by naming their directory, which is what opting out of a downward search leaves.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A candidate that failed to parse was reported and then dropped, so a broad verb
ran everything else and still exited zero: a malformed justfile removed its own
work from CI while the diagnostic scrolled past. Failure now travels back through
resolution and no partial run is started on top of it.
Carrying the recipe rather than just its name also lets arguments be grouped by
what the recipe can actually be called with. Nothing reachable today needs that,
as every verb resolves to a variadic or zero-argument recipe, but the arguments
are paths and a fixed-arity recipe would otherwise fail on arity alone.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Canonicalize justfiles before grouping invocations
misc/just/forward_command.py:393
The dictionary is keyed by the spelling returned for each argument, so one logical justfile is treated as two jobs when arguments mix relative and absolute paths (or aliases through symlinks). For example, paths under pkg and /checkout/pkg produce two invocations of the same variadic recipe, contradicting the run-once guarantee. Key by the resolved justfile while retaining one spelling for invocation.
Normalize paths when applying whole-directory subsumption
misc/just/forward_command.py:398
This lexical string comparison misses equivalent spellings such as ./rust/ql/test versus rust/ql/test. If that whole directory and a child path are both supplied, both reach the recipe even though the comment promises the child will be subsumed, potentially running overlapping tests twice. Compare resolved paths instead.
Run the real-just compatibility tests in CI
misc/just/test_forward_command.py:419
Both compatibility classes are skipped unless just is already on PATH, but the new ubuntu-latest job only runs Bazel and the current runner image does not include just. Consequently CI reports success without executing the tests that validate the hand-written dump fixtures and recipe deduplication against the required tool. Provision a pinned supported just binary to this test or the workflow instead of silently skipping.
Use the repository's actual Rust test path
misc/just/README.md:22
From this repository root, ql/rust/ql/test/{a,b} does not exist; the Rust suite introduced by this PR is at rust/ql/test, as the following sentence itself states. The documented command therefore fails in the standalone checkout.
This issue also appears in the following locations of the same file:
`just` 1.58 or newer is required: recipes forward argument lists using `set lists`, which
is still unstable and did not exist before then. An older one stops with an
`Unknown setting` error pointing at that line, which is clear enough but does not say
which version to move to.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This introduces common "verbs" (
build,test,format,lint,generate) that individual parts of the project can implement, and the shared infrastructure inmisc/just/behind them.This supersedes #19978, which had accumulated too much history to review. Same content, recreated as a readable commit series on top of
main, plus the fixes listed below.Forwarding
A verb is spelled the same everywhere, but what it means is defined per language, next to the code it acts on. The forwarder finds the justfiles implementing a verb for each of its arguments, so there is no central list of who implements what.
Justfiles are looked for in both directions from an argument:
Every distinct recipe found this way runs. Recipes are compared by value, so one reached through
importis recognised as the same job and runs once, while a cross-cutting recipe higher up composes with the more specific ones below instead of hiding them. Arguments are grouped by the recipe they resolve to, sojust build rust javaworks.Two things keep the search useful:
explicit_verbsto stay out of reach of a verb aimed at one of its parents. QL test suites use this:just test .no longer sweeps whole language suites, which take a long time. A verb that passes over such a directory says so and names it._root_<verb>. That is how the root formats bazel files, which belong to no single language, whilejust format cppstays withincpp.Running QL tests
codeqlstandalone--all-checks, abbreviated+, adds the extra checks CI runs, configured per language--codeql=builtskips the build step, consistent with the same pytest optionWhat changed since #19978
The reason for recreating rather than squashing:
set lists(casey/just#1988, now fixed) replaces the whitespace-separated string blobs that the old version used to encode recipe arguments, along with the re-splitting in the Python helpers. That encoding could not carry an argument faithfully: one containing a space was silently split in two, and a value that was set but empty could not be written at all, which is why the Kotlin tests spelledCODEQL_EXTRACTOR_KOTLIN_DIAGNOSTIC_LIMITwith a literal space, to leave something for the split to find.Bugs fixed along the way:
go language-tests-386is restoredRAM_PER_THREADand the nolang search path are passed throughCaveats
When one verb runs several recipes, any non-positional argument has to be understood by all of them. This works for options like
--learnor--codeql, which the shared test definitions all accept.Companion PR
The internal repo companion PR ports all languages to this infrastructure and updates CI accordingly.