Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ Everything here is best-effort. A missing artifact, an expired one, or a token w

Because artifacts are readable and writable from every trigger, all paths behave the same: an automatic run, a `/codeboarding` command and a manual dispatch each reuse the previous analysis and publish their own. A stored analysis is discarded, and the head re-derived from the base, whenever the pinned CodeBoarding version, `.codeboardingignore`, the model selection, the analysis depth or the merge base changes — or when the base graph it grew from is not the one this run compares against.

With no analysis to continue from, a run still reuses the static analysis (the language servers' call graph, `static_analysis.pkl`) saved with the nearest base for the commit analysed or one of its last 100 first-parent ancestors, whatever configuration made it, and the engine brings it up to date. The engine checks that file itself and rebuilds one from another engine cache version, so a new CodeBoarding release or a changed model or depth costs the language-server pass only when it has to. Set `fresh_analysis: 'true'` to rebuild everything, static analysis included.
With no analysis to continue from, as after a new CodeBoarding release, a run still takes two things from the nearest base saved for the commit analysed or one of its last 100 first-parent ancestors, whatever configuration made it. Its component names and descriptions carry over to the components the new analysis continues, so names someone fixed are not lost. Its static analysis (the language servers' call graph, `static_analysis.pkl`) is brought up to date instead of rebuilt; the engine checks that file itself and rebuilds one from another engine cache version, so a release or a changed model or depth costs the language-server pass only when it has to. Every full analysis keeps both from the analysis it replaces. Set `fresh_analysis: 'true'` to rebuild everything, names and static analysis included.

Fork pull requests never carry an analysis forward. They are reviewed on request, each review starts from the base, and nothing they produce is read by a run on this repository's own code: untrusted code must not shape state that a later run loads. The action also accepts `pull_request_target`, which runs on the base branch ref and lets both share one chain; that trigger has its own trade-offs (a PR that adds this workflow will not run it until merged, and the fork gate becomes load-bearing), so `pull_request` remains the recommended default.

Expand Down
2 changes: 1 addition & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ inputs:
included, is built from scratch. A fresh review republishes only the base at its
merge base; later syncs still continue from the analysis branch. Off by default, so
a run continues from the nearest saved analysis and, without one, still reuses the
nearest saved static analysis.
component names and static analysis of the nearest one saved under any configuration.
required: false
default: 'false'
review_guide_url:
Expand Down
4 changes: 3 additions & 1 deletion scripts/action/analyze.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,9 @@ fresh=""
if [ "${FRESH_ANALYSIS:-false}" = true ] && [ "$role" != head ]; then fresh=--fresh; fi

mkdir -p "$state"
if [ -z "$fresh" ] && compatible_state "$state"; then
# A seed from another configuration only lends names and static analysis; it
# holds no engine state to continue from.
if [ -z "$fresh" ] && [ "${SEED_NAMES_ONLY:-false}" != true ] && compatible_state "$state"; then
run_core incremental
if [ "$ANALYSIS_MODE" != incremental ] || { [ "$REQUIRES_FULL" != true ] && [ ! -f "$ANALYSIS_PATH" ]; }; then
fail "Core returned an invalid incremental-analysis result."
Expand Down
63 changes: 42 additions & 21 deletions scripts/action/codeboarding-baseline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,18 @@ baseline_fail() {
exit 1
}

# Reading an entry made under any configuration (CFG_HASH='*') is best effort: it
# only lends names and static analysis, so a failure falls through to the next
# source the way a missing artifact does. Under this run's configuration it fails
# the run. Callers return 1 after it.
baseline_read_fail() {
[ "${CFG_HASH:-}" = "*" ] || baseline_fail "$1"
echo "::warning::$1 Not taking anything from it."
}

# Returns 0 when branch $2 (BASELINE_BRANCH by default) exists in repository $1 and
# 2 when it does not. Any other failure is retried once, so one network blip does
# not fail the run.
# not fail the run, and then returns 1 if baseline_read_fail lets it.
baseline_branch_exists() {
local repository="$1" branch="${2:-$BASELINE_BRANCH}" auth status
auth="$(printf 'x-access-token:%s' "${GIT_TOKEN:-}" | base64 -w0)"
Expand All @@ -31,7 +40,8 @@ baseline_branch_exists() {
esac
[ "$attempt" = 2 ] || sleep "${BASELINE_RETRY_DELAY:-2}"
done
baseline_fail "Could not read $branch from $repository. Check that github_token can read the repository."
baseline_read_fail "Could not read $branch from $repository. Check that github_token can read the repository."
return 1
}

# ---- Reading ----
Expand All @@ -51,13 +61,13 @@ codeboarding_baseline_index() {
awk 'NF >= 2 {print $1, $2, (NF >= 3 ? $3 : "-")}'
}

# Seeds $3 from the branch's entry for $2, or for its nearest first-parent
# ancestor, made under this run's configuration: the trailer pins it the way the
# name pins a saved artifact, and without a configuration hash no entry is
# trusted. Keeps checkout $4's user configuration. Sets SEED_SHA to the commit
# the seed describes.
from_codeboarding_baseline() {
local repository="$1" tip="$2" state="$3" config_from="$4" index commit entry="" scratch="$RUNNER_TEMP/codeboarding-baseline-restore"
# Extracts into $3/.codeboarding the branch's entry for $2, or for its nearest
# first-parent ancestor, made under this run's configuration: the trailer pins it
# the way the name pins a saved artifact, and without a configuration hash no entry
# is trusted. CFG_HASH='*' takes an entry made under any configuration. Sets
# BASELINE_SOURCE to the commit the entry describes.
restore_codeboarding_baseline() {
local repository="$1" tip="$2" dest="$3" index commit entry=""
local status=0
[ -n "${BASELINE_BRANCH:-}" ] || return 1
[ -n "${CFG_HASH:-}" ] || return 1
Expand All @@ -68,37 +78,48 @@ from_codeboarding_baseline() {
echo "::notice::$BASELINE_BRANCH does not exist yet; the first sync creates it."
return 1
fi
[ "$status" = 0 ] || return 1
index="$(codeboarding_baseline_index "$repository")" ||
baseline_fail "Could not fetch $BASELINE_BRANCH from $repository."
{ baseline_read_fail "Could not fetch $BASELINE_BRANCH from $repository."; return 1; }
if [ -z "$index" ]; then
# A code branch, as when a review into another branch names the one that holds the analysis.
echo "::warning::$BASELINE_BRANCH is not a CodeBoarding analysis branch, so its analysis is not read here."
return 1
fi
fetch_commit "$repository" "$tip" "$(( CATCHUP_BOUND + 1 ))" ||
baseline_fail "Could not fetch the history of ${tip:0:7} to look for its analysis."
{ baseline_read_fail "Could not fetch the history of ${tip:0:7} to look for its analysis."; return 1; }
for commit in $(git -C "$CHECKOUT_DIR" rev-list --first-parent --max-count=$(( CATCHUP_BOUND + 1 )) "$tip" 2>/dev/null); do
entry="$(awk -v source="$commit" -v cfg="$CFG_HASH" '$2 == source && $3 == cfg {print $1; exit}' <<< "$index")"
entry="$(awk -v source="$commit" -v cfg="$CFG_HASH" '$2 == source && (cfg == "*" || $3 == cfg) {print $1; exit}' <<< "$index")"
[ -z "$entry" ] || break
done
if [ -z "$entry" ]; then
echo "::notice::$BASELINE_BRANCH has no analysis of ${tip:0:7} or its last $CATCHUP_BOUND ancestors made with this configuration."
[ "$CFG_HASH" = "*" ] ||
echo "::notice::$BASELINE_BRANCH has no analysis of ${tip:0:7} or its last $CATCHUP_BOUND ancestors made with this configuration."
return 1
fi
# The entry is known to exist from here on, so failing to read it is an error.
fetch_commit "$repository" "$entry" ||
baseline_fail "Could not fetch the analysis ${entry:0:7} from $BASELINE_BRANCH."
rm -rf "$scratch"
mkdir -p "$scratch"
git -C "$CHECKOUT_DIR" archive "$entry" .codeboarding | tar -x -C "$scratch" ||
baseline_fail "Could not extract the analysis ${entry:0:7} from $BASELINE_BRANCH."
{ baseline_read_fail "Could not fetch the analysis ${entry:0:7} from $BASELINE_BRANCH."; return 1; }
rm -rf "$dest"
mkdir -p "$dest"
git -C "$CHECKOUT_DIR" archive "$entry" .codeboarding | tar -x -C "$dest" ||
{ baseline_read_fail "Could not extract the analysis ${entry:0:7} from $BASELINE_BRANCH."; return 1; }
# source.json is provenance, not engine state.
rm -f "$scratch/.codeboarding/source.json"
rm -f "$dest/.codeboarding/source.json"
BASELINE_SOURCE="$commit"
}

# Seeds $3 from the branch's entry for $2 or its nearest ancestor (see
# restore_codeboarding_baseline). Keeps checkout $4's user configuration. Sets
# SEED_SHA to the commit the seed describes.
from_codeboarding_baseline() {
local repository="$1" tip="$2" state="$3" config_from="$4" scratch="$RUNNER_TEMP/codeboarding-baseline-restore"
restore_codeboarding_baseline "$repository" "$tip" "$scratch" || return 1
compatible_state "$scratch/.codeboarding" || return 1
replace_state "$scratch/.codeboarding" "$state" "$config_from"
# shellcheck disable=SC2034 # read by the scripts that source this file
SEED_SHA="$commit"
echo "::notice::Starting from the analysis of ${commit:0:7} saved on $BASELINE_BRANCH."
SEED_SHA="$BASELINE_SOURCE"
echo "::notice::Starting from the analysis of ${BASELINE_SOURCE:0:7} saved on $BASELINE_BRANCH."
}

# ---- Writing ----
Expand Down
16 changes: 7 additions & 9 deletions scripts/action/find-ancestor-base.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,19 @@
# under this configuration and downloads it into DEST, so a run without an exact
# base can catch up from it instead of analyzing from scratch. Best effort.
#
# With STATIC_ONLY=true a base saved under any configuration will do, and only
# its static-analysis cache is required, since that is all the caller takes.
# With ANY_CONFIG=true a base saved under any configuration will do: the caller
# takes from it only what outlives a configuration.
#
# Prints ancestor_sha=<the commit whose analysis is now in DEST, or empty> on
# stdout and everything else on stderr.
#
# Needs the tip's first-parent history in CHECKOUT_DIR, at least CATCHUP_BOUND deep.
set -euo pipefail
: "${REPOSITORY:?}" "${TIP_SHA:?}" "${CHECKOUT_DIR:?}" "${DEST:?}"
if [ "${STATIC_ONLY:-false}" = true ]; then
pattern='codeboarding-base-[0-9a-f]{16}-' needs=(static_analysis.pkl static_analysis.sha)
if [ "${ANY_CONFIG:-false}" = true ]; then
pattern='codeboarding-base-[0-9a-f]{16}-'
else
pattern="codeboarding-base-${CFG_HASH:?}-" needs=(analysis.json)
pattern="codeboarding-base-${CFG_HASH:?}-"
fi
BOUND="${CATCHUP_BOUND:-100}"
rm -rf "$DEST"
Expand All @@ -30,7 +30,7 @@ trap report EXIT

# The merge base's first-parent history, nearest first. Distance 0 is the tip
# itself, which a review already looked up by its exact name under this
# configuration. Sync and STATIC_ONLY lookups pass INCLUDE_TIP=true: nothing
# configuration. Sync and ANY_CONFIG lookups pass INCLUDE_TIP=true: nothing
# looked the tip up for them.
walk="$(git -C "$CHECKOUT_DIR" rev-list --first-parent --max-count=$(( BOUND + 1 )) "$TIP_SHA" 2>/dev/null || true)"
[ "${INCLUDE_TIP:-false}" = true ] || walk="$(tail -n +2 <<< "$walk")"
Expand Down Expand Up @@ -77,7 +77,5 @@ done

# Downloaded by its exact name, through the same checks as every other lookup.
GITHUB_OUTPUT="" ARTIFACT_NAME="$name" DEST="$DEST" "$(dirname "$0")/fetch-state.sh" >&2 || exit 0
for file in "${needs[@]}"; do
[ -f "$DEST/$file" ] || exit 0
done
[ -f "$DEST/analysis.json" ] || exit 0
ancestor="${name##*-}"
10 changes: 5 additions & 5 deletions scripts/action/review-generate-baseline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@
# 4. the saved analysis of the nearest ancestor /
# 5. nothing built from scratch
#
# Built from scratch, it still reuses the static analysis of the nearest saved
# base, made under any configuration. With FRESH_ANALYSIS=true nothing saved is
# read (action.yml does not fetch source 1), and the base this run publishes
# replaces the one reviews forking there reused so far.
# Built in full, it still takes the component names and the static analysis of
# the nearest base saved under any configuration. With FRESH_ANALYSIS=true
# nothing saved is read (action.yml does not fetch source 1), and the base this
# run publishes replaces the one reviews forking there reused so far.
#
# Outputs base_analysis_path, and publish_base when this run produced a base
# other pull requests forking there should reuse.
Expand Down Expand Up @@ -50,7 +50,7 @@ else
else
echo "::notice::No saved analysis to start from; analyzing ${REVIEW_BASE_SHA:0:7} from scratch."
keep_user_config "$checkout" "$state"
reuse_static_analysis "$REVIEW_BASE_REPO" "$REVIEW_BASE_SHA" "$state"
reuse_other_configuration "$REVIEW_BASE_REPO" "$REVIEW_BASE_SHA" "$state"
fi
# Core never sees the token.
unset GIT_TOKEN
Expand Down
47 changes: 30 additions & 17 deletions scripts/action/seed-sources.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ source "$HERE/analysis-state.sh"
# shellcheck source=scripts/action/codeboarding-baseline.sh
source "$HERE/codeboarding-baseline.sh"
SEED_SHA=""
# Set by reuse_other_configuration; analyze.sh then goes straight to a full run.
export SEED_NAMES_ONLY=false

# The analysis sync committed under .codeboarding/ on the branch itself. Which
# commit it describes is not recorded, so it is always caught up.
Expand Down Expand Up @@ -67,30 +69,41 @@ from_ancestor_artifact() {
echo "::notice::Starting from the saved analysis of ${ancestor:0:7}."
}

# With no analysis to continue from, the static analysis in the nearest saved base
# for $2 or an ancestor, under any configuration, still spares Core rebuilding it:
# Core refuses one from another cache version itself. Copies both cache files
# into $3 or neither. Best effort.
reuse_static_analysis() {
local repository="$1" tip="$2" state="$3" found ancestor dest="$RUNNER_TEMP/codeboarding-static"
[ "${ANCESTOR_LOOKUP:-false}" = true ] && [ -n "${REPOSITORY:-}" ] && [ -n "$tip" ] || return 0
fetch_commit "$repository" "$tip" "$(( CATCHUP_BOUND + 1 ))" || true
found="$(GH_TOKEN="${GIT_TOKEN:-}" GH_ENTERPRISE_TOKEN="${GIT_TOKEN:-}" TIP_SHA="$tip" DEST="$dest" \
STATIC_ONLY=true INCLUDE_TIP=true CATCHUP_BOUND="$CATCHUP_BOUND" \
"$HERE/find-ancestor-base.sh" || true)"
ancestor="$(awk -F= '$1 == "ancestor_sha" {print $2; exit}' <<< "$found")"
[ -n "$ancestor" ] || return 0
# With no analysis to continue from, as after a release, the nearest base saved
# for $2 or an ancestor under any configuration still lends $3 what outlives a
# configuration: the component names in its analysis.json, which Core keeps for
# the components the full run continues, and its static analysis, which Core
# refuses itself when the cache version changed. Neither is engine state to
# continue from, so analyze.sh runs in full. $3 already holds the analysed
# commit's user configuration. Best effort.
reuse_other_configuration() {
local repository="$1" tip="$2" state="$3" found seed="" source="" dest="$RUNNER_TEMP/codeboarding-other-configuration"
[ -n "$tip" ] || return 0
if [ -n "${CFG_HASH:-}" ] && CFG_HASH='*' restore_codeboarding_baseline "$repository" "$tip" "$dest"; then
seed="$dest/.codeboarding" source="$BASELINE_SOURCE"
elif [ "${ANCESTOR_LOOKUP:-false}" = true ] && [ -n "${REPOSITORY:-}" ]; then
fetch_commit "$repository" "$tip" "$(( CATCHUP_BOUND + 1 ))" || true
found="$(GH_TOKEN="${GIT_TOKEN:-}" GH_ENTERPRISE_TOKEN="${GIT_TOKEN:-}" TIP_SHA="$tip" DEST="$dest" \
ANY_CONFIG=true INCLUDE_TIP=true CATCHUP_BOUND="$CATCHUP_BOUND" \
"$HERE/find-ancestor-base.sh" || true)"
seed="$dest" source="$(awk -F= '$1 == "ancestor_sha" {print $2; exit}' <<< "$found")"
fi
[ -n "$source" ] || return 0
if cp "$seed/analysis.json" "$state/"; then
SEED_NAMES_ONLY=true
echo "::notice::Component names carry over from the analysis of ${source:0:7}, saved under another configuration."
fi
# Core reads the ignore rules from the checkout and keeps a cached file whatever
# they say later, so a cache made under other rules would keep newly ignored
# files and miss newly analysed ones. The two commits say which rules applied;
# the bundle cannot, since uploads drop dotfiles.
if ! git -C "$CHECKOUT_DIR" diff --quiet "$ancestor" "$tip" -- .codeboarding/.codeboardingignore; then
echo "::notice::The saved static analysis of ${ancestor:0:7} was made with another .codeboardingignore; not reusing it."
if ! git -C "$CHECKOUT_DIR" diff --quiet "$source" "$tip" -- .codeboarding/.codeboardingignore; then
echo "::notice::The saved static analysis of ${source:0:7} was made with another .codeboardingignore; not reusing it."
return 0
fi
if ! cp "$dest/static_analysis.pkl" "$dest/static_analysis.sha" "$state/"; then
if ! cp "$seed/static_analysis.pkl" "$seed/static_analysis.sha" "$state/" 2>/dev/null; then
rm -f "$state/static_analysis.pkl" "$state/static_analysis.sha"
return 0
fi
echo "::notice::Reusing the saved static analysis of ${ancestor:0:7}."
echo "::notice::Reusing the saved static analysis of ${source:0:7}."
}
6 changes: 3 additions & 3 deletions scripts/action/sync-generate-baseline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
# 1. an entry on the baseline branch for this commit or an ancestor
# 2. the analysis committed on the branch, or on the parent of a commit that removed it
# 3. the saved analysis of this commit or the nearest ancestor
# 4. nothing: analyzed from scratch, reusing the static analysis of the
# nearest saved base, made under any configuration
# 4. nothing: analyzed in full, taking the component names and the static
# analysis of the nearest base saved under any configuration
#
# With FRESH_ANALYSIS=true nothing saved is read.
set -euo pipefail
Expand All @@ -32,7 +32,7 @@ elif ! from_codeboarding_baseline "${REPOSITORY:-}" "$head_sha" "$state" "$CHECK
! from_parent_committed_baseline "${REPOSITORY:-}" "$head_sha" "$state" "$CHECKOUT_DIR" &&
! from_ancestor_artifact "${REPOSITORY:-}" "$head_sha" "$state" true "$CHECKOUT_DIR"; then
keep_user_config "$CHECKOUT_DIR" "$state"
reuse_static_analysis "${REPOSITORY:-}" "$head_sha" "$state"
reuse_other_configuration "${REPOSITORY:-}" "$head_sha" "$state"
fi

# Core never sees the token.
Expand Down
Loading
Loading