diff --git a/CHANGELOG.md b/CHANGELOG.md index 553c68b..1a0a60c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,18 @@ All notable changes to this project are recorded here. The format follows Keep a ## [Unreleased] +## [0.7.0] - 2026-09-16 + +### Added + +- Prefix locks (#6). A path ending in `/` is a prefix: `claim --job build dist/` covers every path under `dist/`, so `with --job build dist/ -- make` now protects what it names. A claim on a path under a live prefix held by another job is refused `via` the prefix; a claim on a prefix over a live lock under it is refused `via` that path; `check` reports the same `via`. Expired locks in the way are evicted as before. The same job may claim under its own prefix. `dist` without the slash is the directory entry itself, a different key, and is not covered. +- Overlap inside one `batch` is decided while planning, before any transaction: two records of different jobs may not claim a prefix and a path under it (the records are not in each other's snapshot, so neither the ancestor verify nor the descendant scan can see the other). The loser is a `duplicate` refusal naming the path and the record that covers it, and the batch lands nothing. One job may still hold a prefix and a path under it. +- How the race closes: prefixes above a wanted path are verified inside the transaction (absent, or unchanged), and every claim moves a directory token ref (`refs/locks/dirs/`) for each directory above its paths by compare-and-swap from the value its snapshot saw. A prefix claim's scan of what is under it and a path claim's check of what is above it therefore cannot both be stale: one of the two transactions fails and re-plans with the other in view. Both orders are forced in the suite with the before-commit gate. The cost is one extra ref transition per directory level on every claim, and two claims under one directory can now collide once and re-plan; `doctor` knows the token refs. + +### Changed + +- A trailing slash is no longer stripped by normalisation; it is the prefix marker. Before 0.7.0, `dir/file/` named the same key as `dir/file`; now it asks about, or claims, everything under `dir/file`. Every other normalisation rule is unchanged (`dir//` and `./dir/./` are the prefix `dir/`). + ## [0.6.0] - 2026-09-16 ### Added diff --git a/README.md b/README.md index 17f8892..64cdd64 100644 --- a/README.md +++ b/README.md @@ -347,7 +347,7 @@ An outside review of 0.2.1 found the guarantees running ahead of the implementat **What `parent` means.** Ownership plus lifetime, not dependency ordering. A child is admitted only under a live parent held by the same holder. Liveness and holder are checked at planning time; what the generation bump adds at commit time is that the parent's record is unchanged since that check, so a release, a renewal or another child cannot have slipped in between. The bump does not re-check the clock: a parent that expires during the microseconds between planning and commit is still bumped, and its family ends at the next sweep or claim over it. The child is released or swept whenever the parent is, by any command, including a claim that evicts an expired parent. Expiry is not inherited: a child keeps its own `expires`, and a parent's expiry ends the family. Renewing a parent (`extend`) keeps its family. Recreating a job name after its release makes a new record with a fresh family, unrelated to the old one. -**What a path identifies.** The lexical form after normalisation: leading `./`, empty segments, `.` segments and a trailing `/` are removed; absolute paths and `..` are refused. `dir//file`, `dir/./file` and `dir/file/` are one key. Case, symlinks and hard links are not resolved, and `dir/` does not cover `dir/file` (#6). That is a policy, stated, not an omission. +**What a path identifies.** The lexical form after normalisation: leading `./`, empty segments and `.` segments are removed; absolute paths and `..` are refused. `dir//file` and `dir/./file` are one key. Case, symlinks and hard links are not resolved. A trailing `/` is kept and means a prefix: `dir/` covers every path under it, and is covered by any live lock under it, in both directions and inside the transaction (a directory token ref per level, compared-and-swapped by every claim, is what makes a stale scan fail rather than land); `dir` without the slash is the directory entry itself, a different key, and a prefix does not cover it. Before 0.7.0 the slash was stripped; that is the one normalisation rule that changed. **What a lock does not do.** It is a cooperative, time-bounded reservation. `with` claims once, runs, and releases; it does not renew, so the reservation can expire under a long command and another claimant may take the path. Give `--ttl` the command's worst case, or renew with `extend` from inside it. A `check` that says free is an observation, not an admission; the protected write needs a claim. @@ -387,7 +387,7 @@ Output is JSON Lines on every command; there is no text mode. | Command | Does | Stdout line(s) | Exit | |---|---|---|---| | `claim --job --holder [--ttl ] [--note ] ...` | atomically lock the paths for the job; re-claiming with the same job replaces its record; `--note` is one line saying why, carried on every line that names the lock | one `claimed` object with `record`; refusals on stderr | 0 claimed, 1 refused, 2 usage | -| `check ...` | who holds each path, in argument order | one object per path as it is examined | 0 all free, 1 any held | +| `check ...` | who holds each path, in argument order; a path under a live prefix, or a prefix with a live lock under it, is held `via` that other path | one object per path as it is examined | 0 all free, 1 any held | | `list` | every lock, live or expired, with its paths | one object per lock; nothing when empty | 0 | | `sweep` | delete expired locks | one `swept` object per lock, as it goes | 0 | | `store` | the resolved store path | one `store` object | 0 | @@ -413,7 +413,7 @@ Output is JSON Lines on every command; there is no text mode. Every JSON line git-locks writes, on stdout or stderr, matches exactly one definition in [`schema/git-locks.schema.json`](schema/git-locks.schema.json) (JSON Schema 2020-12). `git locks schema` prints that document byte-for-byte, and the test suite validates every line it provokes against it, so the contract cannot drift from the code. Consumers can pin the `$id` URL or the file at a tagged commit. -Paths are repo-relative, `./` prefixes are stripped, and absolute or `..` paths are refused. A path may contain spaces; it may not contain a newline. Job ids match `[A-Za-z0-9][A-Za-z0-9._-]*`. A holder is one line of text; any byte but a newline is stored whole and escaped on output. A note, given with `--note`, is one line saying why the lock is held; it rides on the claim, `show`, `list`, `check` and refusal lines, so the claimant who loses reads the reason and not only the name. A ttl is a decimal number of seconds; a leading zero is not octal. +Paths are repo-relative, `./` prefixes are stripped, and absolute or `..` paths are refused. A path ending in `/` is a prefix and covers everything under it. A path may contain spaces; it may not contain a newline. Job ids match `[A-Za-z0-9][A-Za-z0-9._-]*`. A holder is one line of text; any byte but a newline is stored whole and escaped on output. A note, given with `--note`, is one line saying why the lock is held; it rides on the claim, `show`, `list`, `check` and refusal lines, so the claimant who loses reads the reason and not only the name. A ttl is a decimal number of seconds; a leading zero is not octal. `GIT_LOCKS_NOW=` fixes the clock, for tests; `GIT_LOCKS_PAUSE_BEFORE_COMMIT=` makes every transaction wait for that file, so tests can force interleavings. Timestamps are epoch seconds. diff --git a/bin/git-locks b/bin/git-locks index 64a8ea8..ff04446 100755 --- a/bin/git-locks +++ b/bin/git-locks @@ -60,7 +60,7 @@ DEFAULT_TTL=14400 SCHEMA='git-locks/1' SEM_SCHEMA='git-locks-sem/1' SLOT_SCHEMA='git-locks-slot/1' -VERSION='0.6.0' +VERSION='0.7.0' RETRIES=200 # a plan refused for a stale expectation is re-read and re-planned this many times NOW_CACHED='' # the clock, read once per invocation by now() @@ -87,13 +87,15 @@ claim lock the paths for the job, atomically; re-claiming with the same job r its record; --parent makes it a child: the parent must be live and held by the same holder, and the child is released or swept with it. The claim line carries the record id of this acquisition. --note is one line saying why, carried on every line that names the lock: a refusal reads - 'held by alice: building the release bundle' instead of just 'held by alice' + 'held by alice: building the release bundle' instead of just 'held by alice'. A path ending + in / is a prefix: dist/ covers every path under it and is covered by any lock under it batch read lock records on stdin (blank-line separated: job:, holder:, ttl:, parent:, note:, paths: then one path per line) and claim them all in one transaction, or none release drop the named jobs' locks and all their descendants, in one transaction; --acquisition releases only if the job's current record belongs to that acquisition (an id that survives extend), --record only if the record oid is exactly that one -check who holds each path, with the seconds left; exit 1 if any is held +check who holds each path, with the seconds left; exit 1 if any is held. A path covered by a prefix + lock, or a prefix with a lock under it, is held via that other path list every lock, live or expired, with its paths and the seconds left sweep delete expired locks, each with its descendants store print the store this directory resolves to @@ -304,10 +306,12 @@ path_error() { # detail -> a usage error line on stderr (the caller returns 2) } normalize_path() { # -> prints the lexical form, or returns 2 with the reason on stderr - # Policy, stated: leading ./, empty segments (//), single-dot segments and a - # trailing / are removed; absolute paths and .. segments are refused; case, - # symlinks and hard links are NOT resolved. dir/ and dir/file are different keys. - local p="$1" part parts=() IFS='/' + # Policy, stated: leading ./, empty segments (//) and single-dot segments are + # removed; absolute paths and .. segments are refused; case, symlinks and hard + # links are NOT resolved. A trailing / is kept: dir/ is a prefix that covers + # every path under it; dir is the directory entry itself, a different key. + local p="$1" part parts=() IFS='/' prefix='' + [[ "${p}" == */ ]] && prefix='/' [[ "${p}" == /* ]] && { path_error "${p}: paths are repo-relative" return 2 @@ -328,7 +332,36 @@ normalize_path() { # -> prints the lexical form, or returns 2 with the reason on path_error 'an empty path' return 2 } - printf '%s' "${parts[*]}" + printf '%s%s' "${parts[*]}" "${prefix}" +} + +is_prefix() { [[ "$1" == */ ]]; } # a normalised path that names everything under it + +covers() { # a b -> 0 when a is a prefix lock holding b (never itself) + is_prefix "$1" && [[ "$2" == "$1"?* ]] +} + +ancestors_v() { # VAR path: set VAR to the prefixes above a normalised path, shortest first, newline separated (a/b/c.md -> a/ a/b/; a/b/ -> a/; c.md -> nothing) + local _an_p="$2" _an_acc='' _an_out='' _an_seg + _an_p="${_an_p%/}" + if [[ "${_an_p}" != */* ]]; then + printf -v "$1" '' + return 0 + fi + _an_p="${_an_p%/*}" + while [[ -n "${_an_p}" ]]; do + _an_seg="${_an_p%%/*}" + if [[ "${_an_seg}" == "${_an_p}" ]]; then _an_p=''; else _an_p="${_an_p#*/}"; fi + _an_acc+="${_an_seg}/" + _an_out+="${_an_acc}"$'\n' + done + printf -v "$1" '%s' "${_an_out%$'\n'}" +} + +dir_ref() { # VAR prefix: the directory token ref for a prefix; every claim that touches the directory moves it, so a + local _dr # prefix claim's scan of what is under it and a path claim's check of what is above it cannot both be stale + path_ref _dr "$2" || return 1 + printf -v "$1" '%s/dirs/%s' "${NS}" "${_dr##*/}" } # ---------------------------------------------------------------- the store @@ -441,8 +474,9 @@ snapshot() { test_gate "${GIT_LOCKS_PAUSE_AFTER_READ:-}" # tests force an interleaving between a read and what follows it } -test_gate() { # file-or-empty: when set, wait here until the file exists (at most 30 s); tests only +test_gate() { # file-or-empty: when set, say so in .ready and wait here until the file exists (at most 30 s); tests only [[ -n "$1" ]] || return 0 + : >"$1.ready" # the test waits for this before racing us: a sleep would let the racer win before we had read and planned local waited=0 until [[ -e "$1" ]] || ((waited >= 600)); do sleep 0.05 @@ -582,7 +616,7 @@ plan_lines() { # -> update-ref stdin lines, one per ref, in plan order before="${T_BEFORE[${ref}]}" after="${T_AFTER[${ref}]}" if [[ "${after}" == '=' ]]; then - [[ -n "${before}" ]] && printf 'verify %s %s\n' "${ref}" "${before}" + if [[ -n "${before}" ]]; then printf 'verify %s %s\n' "${ref}" "${before}"; else printf 'verify %s\n' "${ref}"; fi # no old value: the ref must not exist elif [[ -z "${after}" ]]; then [[ -n "${before}" ]] && printf 'delete %s %s\n' "${ref}" "${before}" elif [[ -z "${before}" ]]; then @@ -612,12 +646,16 @@ transact() { # commits the plan; 0 ok, 1 refused (TRANSACT_ERR carries git's wor } # ---------------------------------------------------------------- refusals -refusal() { # path, after describe(): one refusal line on stderr - local _j1 _j2 _j3 +refusal() { # path [via], after describe(): one refusal line on stderr; via names the lock's own path when it differs (a prefix over the path, or a path under a wanted prefix) + local _j1 _j2 _j3 _jv='' json_str _j1 "$1" json_str _j2 "${D_HOLDER}" json_str _j3 "${D_JOB}" - printf '{"event":"refused","path":%s,"holder":%s%s,"job":%s,"expires":%s}\n' "${_j1}" "${_j2}" "${D_NOTE_JSON}" "${_j3}" "${D_EXPIRES}" >&2 + if [[ -n "${2:-}" ]]; then + json_str _jv "$2" + _jv=",\"via\":${_jv}" + fi + printf '{"event":"refused","path":%s,"holder":%s%s,"job":%s%s,"expires":%s}\n' "${_j1}" "${_j2}" "${D_NOTE_JSON}" "${_j3}" "${_jv}" "${D_EXPIRES}" >&2 } parent_refusal() { # child parent detail @@ -627,10 +665,14 @@ parent_refusal() { # child parent detail printf '{"event":"refused","reason":"parent","job":%s,"parent":%s,"detail":"%s"}\n' "${_j1}" "${_j2}" "$3" >&2 } -duplicate_refusal() { # path named twice within one plan - local _j1 +duplicate_refusal() { # path [via]: the path is claimed twice within one plan, or (via) another record of this batch already covers it + local _j1 _jv='' json_str _j1 "$1" - printf '{"event":"refused","reason":"duplicate","path":%s}\n' "${_j1}" >&2 + if [[ -n "${2:-}" ]]; then + json_str _jv "$2" + _jv=",\"via\":${_jv}" + fi + printf '{"event":"refused","reason":"duplicate","path":%s%s}\n' "${_j1}" "${_jv}" >&2 } transaction_refusal() { # git's words, as one line @@ -765,6 +807,7 @@ bump_parent() { # parent-job parent-oid -> plans the parent's blob rewrite wit BATCH_JOBS=() declare -A BATCH_HOLDER=() # job planned in this batch -> holder declare -A BUMPED=() # parent job -> 1 once its family generation is planned in this batch +declare -A BATCH_PATH=() # normalised path planned in this batch -> the job claiming it CONFLICTS=0 CLAIM_LINE='' TERMINATED_PATHS=0 @@ -786,6 +829,22 @@ plan_claim() { # job holder ttl parent note path... -> plans one claim; sets now_v at expires=$((at + ttl)) + # Paths planned earlier in this batch are not in the snapshot, so the checks below cannot see them: a record + # claiming dist/ and another claiming dist/a.js would each plan against a store where the other does not exist, + # and the ancestor verify of one would be absorbed by the create of the other. Overlap inside one batch is + # therefore decided here, and only between different jobs; a job may hold a prefix and a path under it. + local bp bw + for bw in "${wanted[@]}"; do + for bp in "${!BATCH_PATH[@]}"; do + [[ "${BATCH_PATH[${bp}]}" == "${job}" ]] && continue + if covers "${bp}" "${bw}" || covers "${bw}" "${bp}"; then + duplicate_refusal "${bw}" "${bp}" + CONFLICTS=1 + fi + done + done + for bw in "${wanted[@]}"; do BATCH_PATH["${bw}"]="${job}"; done + # The parent, if any: live and the same holder, whether it exists already or is planned earlier in this batch. local pref poid if [[ -n "${parent}" ]]; then @@ -866,6 +925,80 @@ plan_claim() { # job holder ttl parent note path... -> plans one claim; sets fi done + # Prefixes above each wanted path: absent (verified so inside the transaction), the job's own, expired (evicted), + # or another job's live lock, which covers the path. + local w ancs anc aref + for w in "${wanted[@]}"; do + ancestors_v ancs "${w}" + while [[ -n "${ancs}" ]]; do + anc="${ancs%%$'\n'*}" + if [[ "${anc}" == "${ancs}" ]]; then ancs=''; else ancs="${ancs#*$'\n'}"; fi + in_list "${anc}" "${wanted[@]}" && continue # planned above, as one of this claim's own paths + path_ref aref "${anc}" + cur="$(ref_oid "${aref}")" + if [[ -z "${cur}" ]]; then + plan_set "${aref}" '' '=' || fail "${PLAN_CONFLICT}" 1 + continue + fi + field_v rjob "${cur}" job + field_v rexp "${cur}" expires + if [[ "${rjob}" == "${job}" ]]; then + continue + elif [[ -n "${rexp}" && "${rexp}" -le "${at}" ]]; then + in_list "${rjob}" "${evict[@]}" || evict+=("${rjob}") + else + describe "${cur}" + refusal "${w}" "${anc}" + CONFLICTS=1 + fi + done + done + + # Paths under each wanted prefix, from the snapshot: another job's live lock covers the prefix; an expired one is evicted. + # The directory token below makes a stale scan fail at commit. + local rows roid rpaths rp + for w in "${wanted[@]}"; do + is_prefix "${w}" || continue + rows="$(job_refs)" + while IFS=' ' read -r ref roid; do + [[ -z "${ref}" ]] && continue + field_v rjob "${roid}" job + [[ "${rjob}" == "${job}" ]] && continue + record_paths_v rpaths "${roid}" + while [[ -n "${rpaths}" ]]; do + rp="${rpaths%%$'\n'*}" + if [[ "${rp}" == "${rpaths}" ]]; then rpaths=''; else rpaths="${rpaths#*$'\n'}"; fi + [[ -n "${rp}" && "${rp}" == "${w}"?* ]] || continue + field_v rexp "${roid}" expires + if [[ -n "${rexp}" && "${rexp}" -le "${at}" ]]; then + in_list "${rjob}" "${evict[@]}" || evict+=("${rjob}") + else + describe "${roid}" + refusal "${w}" "${rp}" + CONFLICTS=1 + fi + break # one path under the prefix is enough to decide about this record + done + done <<<"${rows}" + done + + # Directory tokens: one per prefix above each wanted path, and the wanted prefix itself. Moved by compare-and-swap + # from the value this snapshot saw to this record, so two claims whose scans could not see each other cannot both + # commit. In a batch the token moves once, to the first record that touches it. + local dref dcur + for w in "${wanted[@]}"; do + ancestors_v ancs "${w}" + is_prefix "${w}" && ancs+="${ancs:+$'\n'}${w}" + while [[ -n "${ancs}" ]]; do + anc="${ancs%%$'\n'*}" + if [[ "${anc}" == "${ancs}" ]]; then ancs=''; else ancs="${ancs#*$'\n'}"; fi + dir_ref dref "${anc}" + [[ -n "${T_BEFORE[${dref}]+x}" && "${T_AFTER[${dref}]}" != '=' ]] && continue + dcur="$(ref_oid "${dref}")" + plan_set "${dref}" "${dcur}" "${new_oid}" || fail "${PLAN_CONFLICT}" 1 + done + done + # The job's own ref, and paths it held before but no longer lists. if [[ -n "${old_job_oid}" ]]; then plan_set "${jref}" "${old_job_oid}" "${new_oid}" || fail "${PLAN_CONFLICT}" 1 @@ -931,22 +1064,27 @@ ref_path() { # oid ref -> which of the record's paths hashes to this ref (for na return 0 } -commit_plan() { # -> 0 committed; 1 lost a race (refusals printed) - transact && return 0 - local lost=0 ref cur p - for ref in "${PLAN_ORDER[@]}"; do - [[ "${ref}" == "${NS}/paths/"* ]] || continue - cur="$(ref_oid "${ref}")" - [[ -z "${cur}" ]] && continue - p="$(ref_path "${cur}" "${ref}")" - [[ -z "${p}" ]] && continue - describe "${cur}" - in_list "${D_JOB}" "${BATCH_JOBS[@]}" && continue - refusal "${p}" - lost=1 +claim_reset() { # planning state for one attempt at a claim or a batch + plan_reset + BATCH_JOBS=() + BATCH_HOLDER=() + BUMPED=() + BATCH_PATH=() + CONFLICTS=0 +} + +commit_claims() { # plan-fn -> 0 committed; exits 1 refused. plan-fn plans every claim of this command against the current snapshot and sets CONFLICTS + local attempt + for ((attempt = 0; attempt < RETRIES; attempt++)); do + ((attempt > 0)) && snapshot # a lost transaction: read again and plan again, so the refusal names what actually won + claim_reset + "$1" + ((CONFLICTS)) && exit 1 + transact && return 0 + sleep 0.01 done - ((lost == 0)) && transaction_refusal - return 1 + transaction_refusal + exit 1 } claim_args() { # parses claim arguments into CA_JOB CA_HOLDER CA_TTL CA_PARENT CA_NOTE CA_PATHS @@ -1003,20 +1141,42 @@ claim_args() { # parses claim arguments into CA_JOB CA_HOLDER CA_TTL CA_PARENT C ((${#CA_PATHS[@]} > 0)) || usage } +plan_one_claim() { plan_claim "${CA_JOB}" "${CA_HOLDER}" "${CA_TTL}" "${CA_PARENT}" "${CA_NOTE}" "${CA_PATHS[@]}"; } + cmd_claim() { claim_args "$@" - plan_reset - plan_claim "${CA_JOB}" "${CA_HOLDER}" "${CA_TTL}" "${CA_PARENT}" "${CA_NOTE}" "${CA_PATHS[@]}" - ((CONFLICTS)) && exit 1 - commit_plan || exit 1 + commit_claims plan_one_claim printf '%s\n' "${CLAIM_LINE}" } # ---------------------------------------------------------------- batch +B_JOB=() +B_HOLDER=() +B_TTL=() +B_PARENT=() +B_NOTE=() +B_PATHS=() # newline joined +B_LINES=() + +plan_batch() { # plans every parsed record against the current snapshot; B_LINES collects the claim lines + local i paths p list + B_LINES=() + for i in "${!B_JOB[@]}"; do + list=() + paths="${B_PATHS[${i}]}" + while [[ -n "${paths}" ]]; do + p="${paths%%$'\n'*}" + if [[ "${p}" == "${paths}" ]]; then paths=''; else paths="${paths#*$'\n'}"; fi + [[ -n "${p}" ]] && list+=("${p}") + done + plan_claim "${B_JOB[${i}]}" "${B_HOLDER[${i}]}" "${B_TTL[${i}]}" "${B_PARENT[${i}]}" "${B_NOTE[${i}]}" "${list[@]}" + B_LINES+=("${CLAIM_LINE}") + done +} + cmd_batch() { (($# == 0)) || usage - local line key val job='' holder='' ttl='' parent='' note='' paths=() in_paths=0 count=0 lines_out=() - plan_reset + local line key val job='' holder='' ttl='' parent='' note='' paths=() in_paths=0 count=0 finish_record() { if [[ -z "${job}" && -z "${holder}" && -z "${ttl}" && -z "${parent}" && -z "${note}" && ${#paths[@]} -eq 0 ]]; then return 0; fi # only a wholly empty record is skipped; one with just parent: or ttl: is malformed [[ -n "${job}" && -n "${holder}" && ${#paths[@]} -gt 0 ]] || fail 'batch: every record needs job:, holder: and at least one path under paths:' 2 @@ -1024,8 +1184,15 @@ cmd_batch() { [[ -z "${ttl}" ]] && ttl="${DEFAULT_TTL}" valid_ttl ttl "${ttl}" || fail 'batch: ttl is a positive number of seconds' 2 valid_holder "${holder}" || fail 'batch: holder must be one line' 2 - plan_claim "${job}" "${holder}" "${ttl}" "${parent}" "${note}" "${paths[@]}" - lines_out+=("${CLAIM_LINE}") + valid_note "${note}" || fail 'batch: note must be one line' 2 + B_JOB+=("${job}") + B_HOLDER+=("${holder}") + B_TTL+=("${ttl}") + B_PARENT+=("${parent}") + B_NOTE+=("${note}") + local joined + joined="$(printf '%s\n' "${paths[@]}")" + B_PATHS+=("${joined}") count=$((count + 1)) job='' holder='' @@ -1059,9 +1226,8 @@ cmd_batch() { done finish_record ((count > 0)) || fail 'batch: no records on stdin' 2 - ((CONFLICTS)) && exit 1 - commit_plan || exit 1 - printf '%s\n' "${lines_out[@]}" + commit_claims plan_batch + printf '%s\n' "${B_LINES[@]}" } # ---------------------------------------------------------------- release @@ -1161,13 +1327,21 @@ cmd_release() { cmd_check() { ensure_snapshot # in this shell, so the $(…) reads below inherit one fresh snapshot instead of each taking their own (($# > 0)) || usage - local at held=0 p n ref cur jp _j1 _j2 + local at held=0 p n ref cur jp _j1 _j2 via jv now_v at for p in "$@"; do n="$(normalize_path "${p}")" || exit 2 path_ref ref "${n}" cur="$(ref_oid "${ref}")" json_str jp "${n}" + via='' + if [[ -z "${cur}" ]] || ! record_live "${cur}" "${at}"; then + covering_v via "${n}" "${at}" # a live prefix above, or a live path under a prefix; empty when none + fi + if [[ -n "${via}" ]]; then + path_ref ref "${via}" + cur="$(ref_oid "${ref}")" + fi if [[ -z "${cur}" ]]; then printf '{"path":%s,"state":"free"}\n' "${jp}" continue @@ -1175,8 +1349,13 @@ cmd_check() { describe "${cur}" json_str _j1 "${D_HOLDER}" json_str _j2 "${D_JOB}" + jv='' + if [[ -n "${via}" ]]; then + json_str jv "${via}" + jv=",\"via\":${jv}" + fi if [[ "${D_EXPIRES}" -gt "${at}" ]]; then - printf '{"path":%s,"state":"held","holder":%s%s,"job":%s,"expires":%s,"remaining":%s}\n' "${jp}" "${_j1}" "${D_NOTE_JSON}" "${_j2}" "${D_EXPIRES}" "${D_REMAINING}" + printf '{"path":%s,"state":"held","holder":%s%s,"job":%s%s,"expires":%s,"remaining":%s}\n' "${jp}" "${_j1}" "${D_NOTE_JSON}" "${_j2}" "${jv}" "${D_EXPIRES}" "${D_REMAINING}" held=1 else printf '{"path":%s,"state":"expired","holder":%s%s,"job":%s,"expires":%s,"remaining":0}\n' "${jp}" "${_j1}" "${D_NOTE_JSON}" "${_j2}" "${D_EXPIRES}" @@ -1184,6 +1363,44 @@ cmd_check() { done return "${held}" } + +record_live() { # oid now -> 0 when the record's expiry is in the future + local _rl + field_v _rl "$1" expires + [[ -n "${_rl}" ]] && ((_rl > $2)) +} + +covering_v() { # VAR path now: set VAR to the path of a live lock that covers this one from above (a prefix) or, for a prefix, from below (a path under it); empty when none + local _cv_p="$2" _cv_at="$3" _cv_ancs _cv_anc _cv_ref _cv_cur _cv_rows _cv_oid _cv_paths _cv_rp + printf -v "$1" '' + ancestors_v _cv_ancs "${_cv_p}" + while [[ -n "${_cv_ancs}" ]]; do + _cv_anc="${_cv_ancs%%$'\n'*}" + if [[ "${_cv_anc}" == "${_cv_ancs}" ]]; then _cv_ancs=''; else _cv_ancs="${_cv_ancs#*$'\n'}"; fi + path_ref _cv_ref "${_cv_anc}" + _cv_cur="${REF_OID[${_cv_ref}]:-}" + if [[ -n "${_cv_cur}" ]] && record_live "${_cv_cur}" "${_cv_at}"; then + printf -v "$1" '%s' "${_cv_anc}" + return 0 + fi + done + is_prefix "${_cv_p}" || return 0 + _cv_rows="$(job_refs)" + while IFS=' ' read -r _cv_ref _cv_oid; do + [[ -z "${_cv_ref}" ]] && continue + record_live "${_cv_oid}" "${_cv_at}" || continue + record_paths_v _cv_paths "${_cv_oid}" + while [[ -n "${_cv_paths}" ]]; do + _cv_rp="${_cv_paths%%$'\n'*}" + if [[ "${_cv_rp}" == "${_cv_paths}" ]]; then _cv_paths=''; else _cv_paths="${_cv_paths#*$'\n'}"; fi + if [[ -n "${_cv_rp}" && "${_cv_rp}" == "${_cv_p}"?* ]]; then + printf -v "$1" '%s' "${_cv_rp}" + return 0 + fi + done + done <<<"${_cv_rows}" + return 0 +} # ---------------------------------------------------------------- list / show / ttl lock_line() { # oid -> one JSON line for list and show; no fork per line, so a list of n locks is O(n) bash and no processes @@ -2022,6 +2239,7 @@ cmd_doctor() { PATHREF_OID["${ref}"]="${oid}" pathrefs+=("${ref}") ;; + "${NS}"/dirs/*) ;; # a directory token: the last record that touched the directory; any object will do, the snapshot already checked it exists "${NS}"/sem/*) rest="${ref#"${NS}"/sem/}" name="${rest%%/*}" @@ -2167,7 +2385,7 @@ cmd_doctor() { cmd_schema() { # the public output schema, one JSON line; the pretty form is schema/git-locks.schema.json in the repository (($# == 0)) || usage cat <<'EOF' -{"$schema":"https://json-schema.org/draft/2020-12/schema","$id":"https://raw.githubusercontent.com/git-stunts/locks/main/schema/git-locks.schema.json","title":"git-locks output","description":"Every line git-locks writes, on stdout or stderr, is one JSON object matching exactly one of the definitions below. Lines are emitted as each result is known (JSON Lines); nothing is buffered into an array. There is no plain-text mode: help is a usage object, errors are error objects, and `git locks schema` prints this document as one line. The one exception: a command wrapped by `with` owns stdout, and git-locks reports around it on stderr.","oneOf":[{"$ref":"#/$defs/check_line"},{"$ref":"#/$defs/list_line"},{"$ref":"#/$defs/claim_line"},{"$ref":"#/$defs/refusal_line"},{"$ref":"#/$defs/release_line"},{"$ref":"#/$defs/sweep_line"},{"$ref":"#/$defs/store_line"},{"$ref":"#/$defs/version_line"},{"$ref":"#/$defs/ttl_line"},{"$ref":"#/$defs/extend_line"},{"$ref":"#/$defs/missing_line"},{"$ref":"#/$defs/sem_line"},{"$ref":"#/$defs/sem_event_line"},{"$ref":"#/$defs/finding_line"},{"$ref":"#/$defs/doctor_line"},{"$ref":"#/$defs/error_line"},{"$ref":"#/$defs/usage_line"}],"$defs":{"job":{"type":"string","pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","description":"The job id, as given to --job. Names the lock: refs/locks/jobs/."},"holder":{"type":"string","minLength":1,"description":"Whoever claimed, as given to --holder. Free text, one line."},"path":{"type":"string","minLength":1,"description":"A repo-relative path, normalised: no leading ./, never absolute, never containing a .. component or a newline."},"epoch":{"type":"integer","minimum":0,"description":"Seconds since the Unix epoch, UTC."},"check_line":{"type":"object","description":"One line per path from `git locks check`, in argument order, as each is examined.","required":["path","state"],"properties":{"path":{"$ref":"#/$defs/path"},"state":{"enum":["free","held","expired"]},"holder":{"$ref":"#/$defs/holder"},"note":{"$ref":"#/$defs/note"},"job":{"$ref":"#/$defs/job"},"expires":{"$ref":"#/$defs/epoch"},"remaining":{"$ref":"#/$defs/remaining"}},"additionalProperties":false,"if":{"properties":{"state":{"const":"free"}}},"then":{"required":["path","state"],"not":{"anyOf":[{"required":["holder"]},{"required":["job"]},{"required":["expires"]},{"required":["remaining"]},{"required":["note"]}]}},"else":{"required":["path","state","holder","job","expires","remaining"]}},"list_line":{"type":"object","description":"One line per lock from `git locks list`, live or expired, in ref order; also the single line of `git locks show --job `.","required":["job","holder","state","claimed","expires","remaining","paths","record","acquisition"],"properties":{"job":{"$ref":"#/$defs/job"},"holder":{"$ref":"#/$defs/holder"},"note":{"$ref":"#/$defs/note"},"state":{"enum":["live","expired"]},"claimed":{"$ref":"#/$defs/epoch"},"expires":{"$ref":"#/$defs/epoch"},"paths":{"type":"array","items":{"$ref":"#/$defs/path"},"uniqueItems":true},"remaining":{"$ref":"#/$defs/remaining"},"parent":{"$ref":"#/$defs/job","description":"Present when the lock is a child: it is released or swept with this job."},"record":{"$ref":"#/$defs/record"},"acquisition":{"$ref":"#/$defs/acquisition"}},"additionalProperties":false},"claim_line":{"type":"object","description":"The single stdout line of a successful `git locks claim`; one per record for `git locks batch`.","required":["event","job","holder","claimed","expires","paths","record","acquisition"],"properties":{"event":{"const":"claimed"},"job":{"$ref":"#/$defs/job"},"holder":{"$ref":"#/$defs/holder"},"note":{"$ref":"#/$defs/note"},"claimed":{"$ref":"#/$defs/epoch"},"expires":{"$ref":"#/$defs/epoch"},"paths":{"type":"array","items":{"$ref":"#/$defs/path"},"minItems":1,"uniqueItems":true},"parent":{"$ref":"#/$defs/job"},"record":{"$ref":"#/$defs/record"},"acquisition":{"$ref":"#/$defs/acquisition"}},"additionalProperties":false},"refusal_line":{"type":"object","description":"On stderr, one line per problem when a claim or batch is refused (exit 1): a held path naming its holder; a parent that is missing, expired or another holder's; a path named by two records of one batch; or a transaction failure with git's message. For semaphores: capacity (full), exists (create), live (delete with live slots).","oneOf":[{"required":["event","path","holder","job","expires"],"properties":{"event":{"const":"refused"},"path":{"$ref":"#/$defs/path"},"holder":{"$ref":"#/$defs/holder"},"note":{"$ref":"#/$defs/note"},"job":{"$ref":"#/$defs/job"},"expires":{"$ref":"#/$defs/epoch"}},"additionalProperties":false},{"required":["event","reason","detail"],"properties":{"event":{"const":"refused"},"reason":{"const":"transaction"},"detail":{"type":"string"}},"additionalProperties":false},{"required":["event","reason","job","parent","detail"],"properties":{"event":{"const":"refused"},"reason":{"const":"parent"},"job":{"$ref":"#/$defs/job"},"parent":{"$ref":"#/$defs/job"},"detail":{"enum":["missing","expired","holder"],"description":"Why the parent cannot be used: no such lock, it has expired, or it belongs to another holder."}},"additionalProperties":false},{"required":["event","reason","path"],"properties":{"event":{"const":"refused"},"reason":{"const":"duplicate"},"path":{"$ref":"#/$defs/path"}},"additionalProperties":false},{"required":["event","reason","semaphore","capacity","live"],"properties":{"event":{"const":"refused"},"reason":{"const":"capacity"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"capacity":{"type":"integer","minimum":1},"live":{"type":"integer","minimum":0,"description":"Slots held by unexpired jobs."}},"additionalProperties":false},{"required":["event","reason","semaphore"],"properties":{"event":{"const":"refused"},"reason":{"const":"exists"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."}},"additionalProperties":false},{"required":["event","reason","semaphore","live"],"properties":{"event":{"const":"refused"},"reason":{"const":"live"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"live":{"type":"integer","minimum":0,"description":"Slots held by unexpired jobs."}},"additionalProperties":false}]},"release_line":{"type":"object","description":"One stdout line per --job of `git locks release`: what was released (with any descendants), or that there was nothing to release.","oneOf":[{"required":["event","job","paths"],"properties":{"event":{"const":"released"},"job":{"$ref":"#/$defs/job"},"paths":{"type":"integer","minimum":0,"description":"How many path refs were deleted, descendants included."},"cascaded":{"type":"array","items":{"$ref":"#/$defs/job"},"description":"Descendant jobs released in the same transaction, sorted; absent when there were none."}},"additionalProperties":false},{"required":["event","job"],"properties":{"event":{"const":"nothing"},"job":{"$ref":"#/$defs/job"},"reason":{"const":"superseded","description":"Present when --acquisition or --record named an acquisition the job no longer holds: nothing was released."}},"additionalProperties":false}]},"sweep_line":{"type":"object","description":"One stdout line per expired lock `git locks sweep` deleted; on stderr, one per lock it could not delete because it changed underneath.","oneOf":[{"required":["event","job","holder","expires"],"properties":{"event":{"const":"swept"},"job":{"$ref":"#/$defs/job"},"holder":{"$ref":"#/$defs/holder"},"expires":{"$ref":"#/$defs/epoch"},"cascaded":{"type":"array","items":{"$ref":"#/$defs/job"},"description":"Descendant jobs swept with this expired parent, sorted; absent when there were none."}},"additionalProperties":false},{"required":["event","job","reason"],"properties":{"event":{"const":"skipped"},"job":{"$ref":"#/$defs/job"},"reason":{"const":"changed underneath"}},"additionalProperties":false}]},"store_line":{"type":"object","description":"The single line of `git locks store`: the absolute path of the store this repository resolves to.","required":["store"],"properties":{"store":{"type":"string","minLength":1}},"additionalProperties":false},"version_line":{"type":"object","description":"The single line of `git locks version`.","required":["name","version"],"properties":{"name":{"const":"git-locks"},"version":{"type":"string","pattern":"^[0-9]+\\.[0-9]+\\.[0-9]+$"}},"additionalProperties":false},"remaining":{"type":"integer","minimum":0,"description":"Seconds until the lock expires, 0 once it has."},"ttl_line":{"type":"object","description":"The single line of `git locks ttl --job `.","required":["job","expires","remaining"],"properties":{"job":{"$ref":"#/$defs/job"},"expires":{"$ref":"#/$defs/epoch"},"remaining":{"$ref":"#/$defs/remaining"}},"additionalProperties":false},"extend_line":{"type":"object","description":"The single line of `git locks extend --job --ttl `: the new expiry.","required":["event","job","expires"],"properties":{"event":{"const":"extended"},"job":{"$ref":"#/$defs/job"},"expires":{"$ref":"#/$defs/epoch"}},"additionalProperties":false},"missing_line":{"type":"object","description":"On stderr, from show, ttl or extend, when no lock exists for the job (exit 1).","required":["event","job"],"properties":{"event":{"const":"missing"},"job":{"$ref":"#/$defs/job"}},"additionalProperties":false},"sem_line":{"type":"object","description":"One line per semaphore from `git locks sem list`, and the single line of `git locks sem show `: capacity, live count, and the live slots.","required":["semaphore","capacity","live","slots"],"properties":{"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"capacity":{"type":"integer","minimum":1},"live":{"type":"integer","minimum":0,"description":"Slots held by unexpired jobs."},"slots":{"type":"array","items":{"type":"object","required":["job","holder","claimed","expires","remaining","record","acquisition"],"properties":{"job":{"$ref":"#/$defs/job"},"holder":{"$ref":"#/$defs/holder"},"claimed":{"$ref":"#/$defs/epoch"},"expires":{"$ref":"#/$defs/epoch"},"remaining":{"$ref":"#/$defs/remaining"},"record":{"$ref":"#/$defs/record"},"acquisition":{"$ref":"#/$defs/acquisition"}},"additionalProperties":false}}},"additionalProperties":false},"sem_event_line":{"type":"object","description":"Semaphore lifecycle lines: created, acquired (with the live count after), released (likewise), nothing (the job held no slot), deleted.","oneOf":[{"required":["event","semaphore","capacity"],"properties":{"event":{"const":"created"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"capacity":{"type":"integer","minimum":1}},"additionalProperties":false},{"required":["event","semaphore","job","holder","claimed","expires","live","capacity","record","acquisition"],"properties":{"event":{"const":"acquired"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"job":{"$ref":"#/$defs/job"},"holder":{"$ref":"#/$defs/holder"},"claimed":{"$ref":"#/$defs/epoch"},"expires":{"$ref":"#/$defs/epoch"},"live":{"type":"integer","minimum":0,"description":"Slots held by unexpired jobs."},"capacity":{"type":"integer","minimum":1},"record":{"$ref":"#/$defs/record"},"acquisition":{"$ref":"#/$defs/acquisition"}},"additionalProperties":false},{"required":["event","semaphore","job","live","capacity"],"properties":{"event":{"const":"released"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"job":{"$ref":"#/$defs/job"},"live":{"type":"integer","minimum":0,"description":"Slots held by unexpired jobs."},"capacity":{"type":"integer","minimum":1}},"additionalProperties":false},{"required":["event","semaphore","job"],"properties":{"event":{"const":"nothing"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"job":{"$ref":"#/$defs/job"},"reason":{"const":"superseded"}},"additionalProperties":false},{"required":["event","semaphore"],"properties":{"event":{"const":"deleted"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."}},"additionalProperties":false},{"required":["event","semaphore"],"properties":{"event":{"const":"missing"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."}},"additionalProperties":false}]},"record":{"type":"string","pattern":"^[0-9a-f]{40}([0-9a-f]{24})?$","description":"The object id of this acquisition's record: the identity a later release or renewal names."},"error_line":{"type":"object","description":"On stderr: a usage failure (exit 2), a store that could not be read (exit 2; nothing is reported free or held), or a failed operation (exit 1).","required":["event","reason","detail"],"properties":{"event":{"const":"error"},"reason":{"enum":["usage","store-read","failed"]},"detail":{"type":"string"}},"additionalProperties":false},"usage_line":{"type":"object","description":"`git locks help` on stdout (exit 0), ` --help`, or a usage error on stderr (exit 2): the usage text as one string.","required":["event","usage"],"properties":{"event":{"const":"usage"},"usage":{"type":"string"}},"additionalProperties":false},"acquisition":{"type":"string","minLength":1,"description":"The identity of one acquisition. Minted by a claim (and by a re-claim, which is a new acquisition), kept by extend and by a child admission's rewrite of the parent record, so a caller can release the acquisition it made even after renewals. Distinct from record, the oid of the current version of its record."},"finding_line":{"type":"object","description":"One stdout line per invariant `git locks doctor` found broken, as it is found. `check` names the invariant, `subject` the job, ref or semaphore it failed for, `detail` what was seen.","required":["event","check","subject","detail"],"properties":{"event":{"const":"finding"},"check":{"enum":["record-decodes","job-ref-name","path-ref-missing","path-ref-elsewhere","path-ref-orphan","path-ref-stray","parent-missing","parent-expired","parent-holder","family-cycle","sem-meta","sem-gen","sem-record","sem-capacity","unknown-ref"]},"subject":{"type":"string","minLength":1},"detail":{"type":"string","minLength":1}},"additionalProperties":false},"doctor_line":{"type":"object","description":"The last stdout line of `git locks doctor`: the store, the reading basis (how many refs and records one snapshot held, and the clock the liveness checks used), the checks run, the number of finding lines, and the verdict. Exit 0 when healthy, 1 with findings. An unreadable store prints an error line instead and exits 2; it is never reported healthy.","required":["event","store","basis","checks","findings","healthy"],"properties":{"event":{"const":"doctor"},"store":{"type":"string","minLength":1},"basis":{"type":"object","required":["refs","records","now"],"properties":{"refs":{"type":"integer","minimum":0},"records":{"type":"integer","minimum":0},"now":{"$ref":"#/$defs/epoch"}},"additionalProperties":false},"checks":{"type":"array","items":{"enum":["record-decodes","job-ref-name","path-ref-missing","path-ref-elsewhere","path-ref-orphan","path-ref-stray","parent-missing","parent-expired","parent-holder","family-cycle","sem-meta","sem-gen","sem-record","sem-capacity","unknown-ref"]},"minItems":1},"findings":{"type":"integer","minimum":0},"healthy":{"type":"boolean"}},"additionalProperties":false},"note":{"type":"string","minLength":1,"pattern":"^[^\\n\\r]*$","description":"One line saying why the lock is held, given at claim time with --note (or note: in a batch record); absent when none was given. Carried on every line that names the lock."}}} +{"$schema":"https://json-schema.org/draft/2020-12/schema","$id":"https://raw.githubusercontent.com/git-stunts/locks/main/schema/git-locks.schema.json","title":"git-locks output","description":"Every line git-locks writes, on stdout or stderr, is one JSON object matching exactly one of the definitions below. Lines are emitted as each result is known (JSON Lines); nothing is buffered into an array. There is no plain-text mode: help is a usage object, errors are error objects, and `git locks schema` prints this document as one line. The one exception: a command wrapped by `with` owns stdout, and git-locks reports around it on stderr.","oneOf":[{"$ref":"#/$defs/check_line"},{"$ref":"#/$defs/list_line"},{"$ref":"#/$defs/claim_line"},{"$ref":"#/$defs/refusal_line"},{"$ref":"#/$defs/release_line"},{"$ref":"#/$defs/sweep_line"},{"$ref":"#/$defs/store_line"},{"$ref":"#/$defs/version_line"},{"$ref":"#/$defs/ttl_line"},{"$ref":"#/$defs/extend_line"},{"$ref":"#/$defs/missing_line"},{"$ref":"#/$defs/sem_line"},{"$ref":"#/$defs/sem_event_line"},{"$ref":"#/$defs/finding_line"},{"$ref":"#/$defs/doctor_line"},{"$ref":"#/$defs/error_line"},{"$ref":"#/$defs/usage_line"}],"$defs":{"job":{"type":"string","pattern":"^[A-Za-z0-9][A-Za-z0-9._-]*$","description":"The job id, as given to --job. Names the lock: refs/locks/jobs/."},"holder":{"type":"string","minLength":1,"description":"Whoever claimed, as given to --holder. Free text, one line."},"path":{"type":"string","minLength":1,"description":"A repo-relative path, normalised: no leading ./, never absolute, never containing a .. component or a newline. A trailing / makes it a prefix: dist/ covers every path under dist/ (and is covered by any lock under it); dist without the slash is the directory entry itself, a different key."},"epoch":{"type":"integer","minimum":0,"description":"Seconds since the Unix epoch, UTC."},"check_line":{"type":"object","description":"One line per path from `git locks check`, in argument order, as each is examined.","required":["path","state"],"properties":{"path":{"$ref":"#/$defs/path"},"state":{"enum":["free","held","expired"]},"holder":{"$ref":"#/$defs/holder"},"note":{"$ref":"#/$defs/note"},"job":{"$ref":"#/$defs/job"},"via":{"$ref":"#/$defs/via"},"expires":{"$ref":"#/$defs/epoch"},"remaining":{"$ref":"#/$defs/remaining"}},"additionalProperties":false,"if":{"properties":{"state":{"const":"free"}}},"then":{"required":["path","state"],"not":{"anyOf":[{"required":["holder"]},{"required":["job"]},{"required":["expires"]},{"required":["remaining"]},{"required":["note"]},{"required":["via"]}]}},"else":{"required":["path","state","holder","job","expires","remaining"]}},"list_line":{"type":"object","description":"One line per lock from `git locks list`, live or expired, in ref order; also the single line of `git locks show --job `.","required":["job","holder","state","claimed","expires","remaining","paths","record","acquisition"],"properties":{"job":{"$ref":"#/$defs/job"},"holder":{"$ref":"#/$defs/holder"},"note":{"$ref":"#/$defs/note"},"state":{"enum":["live","expired"]},"claimed":{"$ref":"#/$defs/epoch"},"expires":{"$ref":"#/$defs/epoch"},"paths":{"type":"array","items":{"$ref":"#/$defs/path"},"uniqueItems":true},"remaining":{"$ref":"#/$defs/remaining"},"parent":{"$ref":"#/$defs/job","description":"Present when the lock is a child: it is released or swept with this job."},"record":{"$ref":"#/$defs/record"},"acquisition":{"$ref":"#/$defs/acquisition"}},"additionalProperties":false},"claim_line":{"type":"object","description":"The single stdout line of a successful `git locks claim`; one per record for `git locks batch`.","required":["event","job","holder","claimed","expires","paths","record","acquisition"],"properties":{"event":{"const":"claimed"},"job":{"$ref":"#/$defs/job"},"holder":{"$ref":"#/$defs/holder"},"note":{"$ref":"#/$defs/note"},"claimed":{"$ref":"#/$defs/epoch"},"expires":{"$ref":"#/$defs/epoch"},"paths":{"type":"array","items":{"$ref":"#/$defs/path"},"minItems":1,"uniqueItems":true},"parent":{"$ref":"#/$defs/job"},"record":{"$ref":"#/$defs/record"},"acquisition":{"$ref":"#/$defs/acquisition"}},"additionalProperties":false},"refusal_line":{"type":"object","description":"On stderr, one line per problem when a claim or batch is refused (exit 1): a held path naming its holder; a parent that is missing, expired or another holder's; a path named by two records of one batch; or a transaction failure with git's message. For semaphores: capacity (full), exists (create), live (delete with live slots).","oneOf":[{"required":["event","path","holder","job","expires"],"properties":{"event":{"const":"refused"},"path":{"$ref":"#/$defs/path"},"holder":{"$ref":"#/$defs/holder"},"note":{"$ref":"#/$defs/note"},"job":{"$ref":"#/$defs/job"},"via":{"$ref":"#/$defs/via"},"expires":{"$ref":"#/$defs/epoch"}},"additionalProperties":false},{"required":["event","reason","detail"],"properties":{"event":{"const":"refused"},"reason":{"const":"transaction"},"detail":{"type":"string"}},"additionalProperties":false},{"required":["event","reason","job","parent","detail"],"properties":{"event":{"const":"refused"},"reason":{"const":"parent"},"job":{"$ref":"#/$defs/job"},"parent":{"$ref":"#/$defs/job"},"detail":{"enum":["missing","expired","holder"],"description":"Why the parent cannot be used: no such lock, it has expired, or it belongs to another holder."}},"additionalProperties":false},{"required":["event","reason","path"],"properties":{"event":{"const":"refused"},"reason":{"const":"duplicate"},"path":{"$ref":"#/$defs/path"},"via":{"$ref":"#/$defs/via"}},"additionalProperties":false},{"required":["event","reason","semaphore","capacity","live"],"properties":{"event":{"const":"refused"},"reason":{"const":"capacity"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"capacity":{"type":"integer","minimum":1},"live":{"type":"integer","minimum":0,"description":"Slots held by unexpired jobs."}},"additionalProperties":false},{"required":["event","reason","semaphore"],"properties":{"event":{"const":"refused"},"reason":{"const":"exists"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."}},"additionalProperties":false},{"required":["event","reason","semaphore","live"],"properties":{"event":{"const":"refused"},"reason":{"const":"live"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"live":{"type":"integer","minimum":0,"description":"Slots held by unexpired jobs."}},"additionalProperties":false}]},"release_line":{"type":"object","description":"One stdout line per --job of `git locks release`: what was released (with any descendants), or that there was nothing to release.","oneOf":[{"required":["event","job","paths"],"properties":{"event":{"const":"released"},"job":{"$ref":"#/$defs/job"},"paths":{"type":"integer","minimum":0,"description":"How many path refs were deleted, descendants included."},"cascaded":{"type":"array","items":{"$ref":"#/$defs/job"},"description":"Descendant jobs released in the same transaction, sorted; absent when there were none."}},"additionalProperties":false},{"required":["event","job"],"properties":{"event":{"const":"nothing"},"job":{"$ref":"#/$defs/job"},"reason":{"const":"superseded","description":"Present when --acquisition or --record named an acquisition the job no longer holds: nothing was released."}},"additionalProperties":false}]},"sweep_line":{"type":"object","description":"One stdout line per expired lock `git locks sweep` deleted; on stderr, one per lock it could not delete because it changed underneath.","oneOf":[{"required":["event","job","holder","expires"],"properties":{"event":{"const":"swept"},"job":{"$ref":"#/$defs/job"},"holder":{"$ref":"#/$defs/holder"},"expires":{"$ref":"#/$defs/epoch"},"cascaded":{"type":"array","items":{"$ref":"#/$defs/job"},"description":"Descendant jobs swept with this expired parent, sorted; absent when there were none."}},"additionalProperties":false},{"required":["event","job","reason"],"properties":{"event":{"const":"skipped"},"job":{"$ref":"#/$defs/job"},"reason":{"const":"changed underneath"}},"additionalProperties":false}]},"store_line":{"type":"object","description":"The single line of `git locks store`: the absolute path of the store this repository resolves to.","required":["store"],"properties":{"store":{"type":"string","minLength":1}},"additionalProperties":false},"version_line":{"type":"object","description":"The single line of `git locks version`.","required":["name","version"],"properties":{"name":{"const":"git-locks"},"version":{"type":"string","pattern":"^[0-9]+\\.[0-9]+\\.[0-9]+$"}},"additionalProperties":false},"remaining":{"type":"integer","minimum":0,"description":"Seconds until the lock expires, 0 once it has."},"ttl_line":{"type":"object","description":"The single line of `git locks ttl --job `.","required":["job","expires","remaining"],"properties":{"job":{"$ref":"#/$defs/job"},"expires":{"$ref":"#/$defs/epoch"},"remaining":{"$ref":"#/$defs/remaining"}},"additionalProperties":false},"extend_line":{"type":"object","description":"The single line of `git locks extend --job --ttl `: the new expiry.","required":["event","job","expires"],"properties":{"event":{"const":"extended"},"job":{"$ref":"#/$defs/job"},"expires":{"$ref":"#/$defs/epoch"}},"additionalProperties":false},"missing_line":{"type":"object","description":"On stderr, from show, ttl or extend, when no lock exists for the job (exit 1).","required":["event","job"],"properties":{"event":{"const":"missing"},"job":{"$ref":"#/$defs/job"}},"additionalProperties":false},"sem_line":{"type":"object","description":"One line per semaphore from `git locks sem list`, and the single line of `git locks sem show `: capacity, live count, and the live slots.","required":["semaphore","capacity","live","slots"],"properties":{"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"capacity":{"type":"integer","minimum":1},"live":{"type":"integer","minimum":0,"description":"Slots held by unexpired jobs."},"slots":{"type":"array","items":{"type":"object","required":["job","holder","claimed","expires","remaining","record","acquisition"],"properties":{"job":{"$ref":"#/$defs/job"},"holder":{"$ref":"#/$defs/holder"},"claimed":{"$ref":"#/$defs/epoch"},"expires":{"$ref":"#/$defs/epoch"},"remaining":{"$ref":"#/$defs/remaining"},"record":{"$ref":"#/$defs/record"},"acquisition":{"$ref":"#/$defs/acquisition"}},"additionalProperties":false}}},"additionalProperties":false},"sem_event_line":{"type":"object","description":"Semaphore lifecycle lines: created, acquired (with the live count after), released (likewise), nothing (the job held no slot), deleted.","oneOf":[{"required":["event","semaphore","capacity"],"properties":{"event":{"const":"created"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"capacity":{"type":"integer","minimum":1}},"additionalProperties":false},{"required":["event","semaphore","job","holder","claimed","expires","live","capacity","record","acquisition"],"properties":{"event":{"const":"acquired"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"job":{"$ref":"#/$defs/job"},"holder":{"$ref":"#/$defs/holder"},"claimed":{"$ref":"#/$defs/epoch"},"expires":{"$ref":"#/$defs/epoch"},"live":{"type":"integer","minimum":0,"description":"Slots held by unexpired jobs."},"capacity":{"type":"integer","minimum":1},"record":{"$ref":"#/$defs/record"},"acquisition":{"$ref":"#/$defs/acquisition"}},"additionalProperties":false},{"required":["event","semaphore","job","live","capacity"],"properties":{"event":{"const":"released"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"job":{"$ref":"#/$defs/job"},"live":{"type":"integer","minimum":0,"description":"Slots held by unexpired jobs."},"capacity":{"type":"integer","minimum":1}},"additionalProperties":false},{"required":["event","semaphore","job"],"properties":{"event":{"const":"nothing"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."},"job":{"$ref":"#/$defs/job"},"reason":{"const":"superseded"}},"additionalProperties":false},{"required":["event","semaphore"],"properties":{"event":{"const":"deleted"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."}},"additionalProperties":false},{"required":["event","semaphore"],"properties":{"event":{"const":"missing"},"semaphore":{"$ref":"#/$defs/job","description":"The semaphore's name; same grammar as a job id."}},"additionalProperties":false}]},"record":{"type":"string","pattern":"^[0-9a-f]{40}([0-9a-f]{24})?$","description":"The object id of this acquisition's record: the identity a later release or renewal names."},"error_line":{"type":"object","description":"On stderr: a usage failure (exit 2), a store that could not be read (exit 2; nothing is reported free or held), or a failed operation (exit 1).","required":["event","reason","detail"],"properties":{"event":{"const":"error"},"reason":{"enum":["usage","store-read","failed"]},"detail":{"type":"string"}},"additionalProperties":false},"usage_line":{"type":"object","description":"`git locks help` on stdout (exit 0), ` --help`, or a usage error on stderr (exit 2): the usage text as one string.","required":["event","usage"],"properties":{"event":{"const":"usage"},"usage":{"type":"string"}},"additionalProperties":false},"acquisition":{"type":"string","minLength":1,"description":"The identity of one acquisition. Minted by a claim (and by a re-claim, which is a new acquisition), kept by extend and by a child admission's rewrite of the parent record, so a caller can release the acquisition it made even after renewals. Distinct from record, the oid of the current version of its record."},"finding_line":{"type":"object","description":"One stdout line per invariant `git locks doctor` found broken, as it is found. `check` names the invariant, `subject` the job, ref or semaphore it failed for, `detail` what was seen.","required":["event","check","subject","detail"],"properties":{"event":{"const":"finding"},"check":{"enum":["record-decodes","job-ref-name","path-ref-missing","path-ref-elsewhere","path-ref-orphan","path-ref-stray","parent-missing","parent-expired","parent-holder","family-cycle","sem-meta","sem-gen","sem-record","sem-capacity","unknown-ref"]},"subject":{"type":"string","minLength":1},"detail":{"type":"string","minLength":1}},"additionalProperties":false},"doctor_line":{"type":"object","description":"The last stdout line of `git locks doctor`: the store, the reading basis (how many refs and records one snapshot held, and the clock the liveness checks used), the checks run, the number of finding lines, and the verdict. Exit 0 when healthy, 1 with findings. An unreadable store prints an error line instead and exits 2; it is never reported healthy.","required":["event","store","basis","checks","findings","healthy"],"properties":{"event":{"const":"doctor"},"store":{"type":"string","minLength":1},"basis":{"type":"object","required":["refs","records","now"],"properties":{"refs":{"type":"integer","minimum":0},"records":{"type":"integer","minimum":0},"now":{"$ref":"#/$defs/epoch"}},"additionalProperties":false},"checks":{"type":"array","items":{"enum":["record-decodes","job-ref-name","path-ref-missing","path-ref-elsewhere","path-ref-orphan","path-ref-stray","parent-missing","parent-expired","parent-holder","family-cycle","sem-meta","sem-gen","sem-record","sem-capacity","unknown-ref"]},"minItems":1},"findings":{"type":"integer","minimum":0},"healthy":{"type":"boolean"}},"additionalProperties":false},"note":{"type":"string","minLength":1,"pattern":"^[^\\n\\r]*$","description":"One line saying why the lock is held, given at claim time with --note (or note: in a batch record); absent when none was given. Carried on every line that names the lock."},"via":{"$ref":"#/$defs/path","description":"The covering lock's own path when it differs from the one asked about or claimed: the prefix above it, or the path under a wanted prefix. Absent when the lock is on the path itself."}}} EOF } diff --git a/lib/000-prelude.sh b/lib/000-prelude.sh index 5a679d6..535ddab 100644 --- a/lib/000-prelude.sh +++ b/lib/000-prelude.sh @@ -60,7 +60,7 @@ DEFAULT_TTL=14400 SCHEMA='git-locks/1' SEM_SCHEMA='git-locks-sem/1' SLOT_SCHEMA='git-locks-slot/1' -VERSION='0.6.0' +VERSION='0.7.0' RETRIES=200 # a plan refused for a stale expectation is re-read and re-planned this many times NOW_CACHED='' # the clock, read once per invocation by now() @@ -87,13 +87,15 @@ claim lock the paths for the job, atomically; re-claiming with the same job r its record; --parent makes it a child: the parent must be live and held by the same holder, and the child is released or swept with it. The claim line carries the record id of this acquisition. --note is one line saying why, carried on every line that names the lock: a refusal reads - 'held by alice: building the release bundle' instead of just 'held by alice' + 'held by alice: building the release bundle' instead of just 'held by alice'. A path ending + in / is a prefix: dist/ covers every path under it and is covered by any lock under it batch read lock records on stdin (blank-line separated: job:, holder:, ttl:, parent:, note:, paths: then one path per line) and claim them all in one transaction, or none release drop the named jobs' locks and all their descendants, in one transaction; --acquisition releases only if the job's current record belongs to that acquisition (an id that survives extend), --record only if the record oid is exactly that one -check who holds each path, with the seconds left; exit 1 if any is held +check who holds each path, with the seconds left; exit 1 if any is held. A path covered by a prefix + lock, or a prefix with a lock under it, is held via that other path list every lock, live or expired, with its paths and the seconds left sweep delete expired locks, each with its descendants store print the store this directory resolves to diff --git a/lib/030-time-refs-records.sh b/lib/030-time-refs-records.sh index 2808adb..ad316dd 100644 --- a/lib/030-time-refs-records.sh +++ b/lib/030-time-refs-records.sh @@ -52,10 +52,12 @@ path_error() { # detail -> a usage error line on stderr (the caller returns 2) } normalize_path() { # -> prints the lexical form, or returns 2 with the reason on stderr - # Policy, stated: leading ./, empty segments (//), single-dot segments and a - # trailing / are removed; absolute paths and .. segments are refused; case, - # symlinks and hard links are NOT resolved. dir/ and dir/file are different keys. - local p="$1" part parts=() IFS='/' + # Policy, stated: leading ./, empty segments (//) and single-dot segments are + # removed; absolute paths and .. segments are refused; case, symlinks and hard + # links are NOT resolved. A trailing / is kept: dir/ is a prefix that covers + # every path under it; dir is the directory entry itself, a different key. + local p="$1" part parts=() IFS='/' prefix='' + [[ "${p}" == */ ]] && prefix='/' [[ "${p}" == /* ]] && { path_error "${p}: paths are repo-relative" return 2 @@ -76,5 +78,34 @@ normalize_path() { # -> prints the lexical form, or returns 2 with the reason on path_error 'an empty path' return 2 } - printf '%s' "${parts[*]}" + printf '%s%s' "${parts[*]}" "${prefix}" +} + +is_prefix() { [[ "$1" == */ ]]; } # a normalised path that names everything under it + +covers() { # a b -> 0 when a is a prefix lock holding b (never itself) + is_prefix "$1" && [[ "$2" == "$1"?* ]] +} + +ancestors_v() { # VAR path: set VAR to the prefixes above a normalised path, shortest first, newline separated (a/b/c.md -> a/ a/b/; a/b/ -> a/; c.md -> nothing) + local _an_p="$2" _an_acc='' _an_out='' _an_seg + _an_p="${_an_p%/}" + if [[ "${_an_p}" != */* ]]; then + printf -v "$1" '' + return 0 + fi + _an_p="${_an_p%/*}" + while [[ -n "${_an_p}" ]]; do + _an_seg="${_an_p%%/*}" + if [[ "${_an_seg}" == "${_an_p}" ]]; then _an_p=''; else _an_p="${_an_p#*/}"; fi + _an_acc+="${_an_seg}/" + _an_out+="${_an_acc}"$'\n' + done + printf -v "$1" '%s' "${_an_out%$'\n'}" +} + +dir_ref() { # VAR prefix: the directory token ref for a prefix; every claim that touches the directory moves it, so a + local _dr # prefix claim's scan of what is under it and a path claim's check of what is above it cannot both be stale + path_ref _dr "$2" || return 1 + printf -v "$1" '%s/dirs/%s' "${NS}" "${_dr##*/}" } diff --git a/lib/050-the-snapshot.sh b/lib/050-the-snapshot.sh index c4766dc..4fcdc4f 100644 --- a/lib/050-the-snapshot.sh +++ b/lib/050-the-snapshot.sh @@ -81,8 +81,9 @@ snapshot() { test_gate "${GIT_LOCKS_PAUSE_AFTER_READ:-}" # tests force an interleaving between a read and what follows it } -test_gate() { # file-or-empty: when set, wait here until the file exists (at most 30 s); tests only +test_gate() { # file-or-empty: when set, say so in .ready and wait here until the file exists (at most 30 s); tests only [[ -n "$1" ]] || return 0 + : >"$1.ready" # the test waits for this before racing us: a sleep would let the racer win before we had read and planned local waited=0 until [[ -e "$1" ]] || ((waited >= 600)); do sleep 0.05 diff --git a/lib/060-the-transition-plan.sh b/lib/060-the-transition-plan.sh index cfbfd14..9cd7f60 100644 --- a/lib/060-the-transition-plan.sh +++ b/lib/060-the-transition-plan.sh @@ -46,7 +46,7 @@ plan_lines() { # -> update-ref stdin lines, one per ref, in plan order before="${T_BEFORE[${ref}]}" after="${T_AFTER[${ref}]}" if [[ "${after}" == '=' ]]; then - [[ -n "${before}" ]] && printf 'verify %s %s\n' "${ref}" "${before}" + if [[ -n "${before}" ]]; then printf 'verify %s %s\n' "${ref}" "${before}"; else printf 'verify %s\n' "${ref}"; fi # no old value: the ref must not exist elif [[ -z "${after}" ]]; then [[ -n "${before}" ]] && printf 'delete %s %s\n' "${ref}" "${before}" elif [[ -z "${before}" ]]; then diff --git a/lib/070-refusals.sh b/lib/070-refusals.sh index 93f25bd..58c7c2d 100644 --- a/lib/070-refusals.sh +++ b/lib/070-refusals.sh @@ -1,11 +1,15 @@ # ---------------------------------------------------------------- refusals -refusal() { # path, after describe(): one refusal line on stderr - local _j1 _j2 _j3 +refusal() { # path [via], after describe(): one refusal line on stderr; via names the lock's own path when it differs (a prefix over the path, or a path under a wanted prefix) + local _j1 _j2 _j3 _jv='' json_str _j1 "$1" json_str _j2 "${D_HOLDER}" json_str _j3 "${D_JOB}" - printf '{"event":"refused","path":%s,"holder":%s%s,"job":%s,"expires":%s}\n' "${_j1}" "${_j2}" "${D_NOTE_JSON}" "${_j3}" "${D_EXPIRES}" >&2 + if [[ -n "${2:-}" ]]; then + json_str _jv "$2" + _jv=",\"via\":${_jv}" + fi + printf '{"event":"refused","path":%s,"holder":%s%s,"job":%s%s,"expires":%s}\n' "${_j1}" "${_j2}" "${D_NOTE_JSON}" "${_j3}" "${_jv}" "${D_EXPIRES}" >&2 } parent_refusal() { # child parent detail @@ -15,10 +19,14 @@ parent_refusal() { # child parent detail printf '{"event":"refused","reason":"parent","job":%s,"parent":%s,"detail":"%s"}\n' "${_j1}" "${_j2}" "$3" >&2 } -duplicate_refusal() { # path named twice within one plan - local _j1 +duplicate_refusal() { # path [via]: the path is claimed twice within one plan, or (via) another record of this batch already covers it + local _j1 _jv='' json_str _j1 "$1" - printf '{"event":"refused","reason":"duplicate","path":%s}\n' "${_j1}" >&2 + if [[ -n "${2:-}" ]]; then + json_str _jv "$2" + _jv=",\"via\":${_jv}" + fi + printf '{"event":"refused","reason":"duplicate","path":%s%s}\n' "${_j1}" "${_jv}" >&2 } transaction_refusal() { # git's words, as one line diff --git a/lib/090-claim-planning.sh b/lib/090-claim-planning.sh index d07bd0a..a4b45c1 100644 --- a/lib/090-claim-planning.sh +++ b/lib/090-claim-planning.sh @@ -3,6 +3,7 @@ BATCH_JOBS=() declare -A BATCH_HOLDER=() # job planned in this batch -> holder declare -A BUMPED=() # parent job -> 1 once its family generation is planned in this batch +declare -A BATCH_PATH=() # normalised path planned in this batch -> the job claiming it CONFLICTS=0 CLAIM_LINE='' TERMINATED_PATHS=0 @@ -24,6 +25,22 @@ plan_claim() { # job holder ttl parent note path... -> plans one claim; sets now_v at expires=$((at + ttl)) + # Paths planned earlier in this batch are not in the snapshot, so the checks below cannot see them: a record + # claiming dist/ and another claiming dist/a.js would each plan against a store where the other does not exist, + # and the ancestor verify of one would be absorbed by the create of the other. Overlap inside one batch is + # therefore decided here, and only between different jobs; a job may hold a prefix and a path under it. + local bp bw + for bw in "${wanted[@]}"; do + for bp in "${!BATCH_PATH[@]}"; do + [[ "${BATCH_PATH[${bp}]}" == "${job}" ]] && continue + if covers "${bp}" "${bw}" || covers "${bw}" "${bp}"; then + duplicate_refusal "${bw}" "${bp}" + CONFLICTS=1 + fi + done + done + for bw in "${wanted[@]}"; do BATCH_PATH["${bw}"]="${job}"; done + # The parent, if any: live and the same holder, whether it exists already or is planned earlier in this batch. local pref poid if [[ -n "${parent}" ]]; then @@ -104,6 +121,80 @@ plan_claim() { # job holder ttl parent note path... -> plans one claim; sets fi done + # Prefixes above each wanted path: absent (verified so inside the transaction), the job's own, expired (evicted), + # or another job's live lock, which covers the path. + local w ancs anc aref + for w in "${wanted[@]}"; do + ancestors_v ancs "${w}" + while [[ -n "${ancs}" ]]; do + anc="${ancs%%$'\n'*}" + if [[ "${anc}" == "${ancs}" ]]; then ancs=''; else ancs="${ancs#*$'\n'}"; fi + in_list "${anc}" "${wanted[@]}" && continue # planned above, as one of this claim's own paths + path_ref aref "${anc}" + cur="$(ref_oid "${aref}")" + if [[ -z "${cur}" ]]; then + plan_set "${aref}" '' '=' || fail "${PLAN_CONFLICT}" 1 + continue + fi + field_v rjob "${cur}" job + field_v rexp "${cur}" expires + if [[ "${rjob}" == "${job}" ]]; then + continue + elif [[ -n "${rexp}" && "${rexp}" -le "${at}" ]]; then + in_list "${rjob}" "${evict[@]}" || evict+=("${rjob}") + else + describe "${cur}" + refusal "${w}" "${anc}" + CONFLICTS=1 + fi + done + done + + # Paths under each wanted prefix, from the snapshot: another job's live lock covers the prefix; an expired one is evicted. + # The directory token below makes a stale scan fail at commit. + local rows roid rpaths rp + for w in "${wanted[@]}"; do + is_prefix "${w}" || continue + rows="$(job_refs)" + while IFS=' ' read -r ref roid; do + [[ -z "${ref}" ]] && continue + field_v rjob "${roid}" job + [[ "${rjob}" == "${job}" ]] && continue + record_paths_v rpaths "${roid}" + while [[ -n "${rpaths}" ]]; do + rp="${rpaths%%$'\n'*}" + if [[ "${rp}" == "${rpaths}" ]]; then rpaths=''; else rpaths="${rpaths#*$'\n'}"; fi + [[ -n "${rp}" && "${rp}" == "${w}"?* ]] || continue + field_v rexp "${roid}" expires + if [[ -n "${rexp}" && "${rexp}" -le "${at}" ]]; then + in_list "${rjob}" "${evict[@]}" || evict+=("${rjob}") + else + describe "${roid}" + refusal "${w}" "${rp}" + CONFLICTS=1 + fi + break # one path under the prefix is enough to decide about this record + done + done <<<"${rows}" + done + + # Directory tokens: one per prefix above each wanted path, and the wanted prefix itself. Moved by compare-and-swap + # from the value this snapshot saw to this record, so two claims whose scans could not see each other cannot both + # commit. In a batch the token moves once, to the first record that touches it. + local dref dcur + for w in "${wanted[@]}"; do + ancestors_v ancs "${w}" + is_prefix "${w}" && ancs+="${ancs:+$'\n'}${w}" + while [[ -n "${ancs}" ]]; do + anc="${ancs%%$'\n'*}" + if [[ "${anc}" == "${ancs}" ]]; then ancs=''; else ancs="${ancs#*$'\n'}"; fi + dir_ref dref "${anc}" + [[ -n "${T_BEFORE[${dref}]+x}" && "${T_AFTER[${dref}]}" != '=' ]] && continue + dcur="$(ref_oid "${dref}")" + plan_set "${dref}" "${dcur}" "${new_oid}" || fail "${PLAN_CONFLICT}" 1 + done + done + # The job's own ref, and paths it held before but no longer lists. if [[ -n "${old_job_oid}" ]]; then plan_set "${jref}" "${old_job_oid}" "${new_oid}" || fail "${PLAN_CONFLICT}" 1 @@ -169,22 +260,27 @@ ref_path() { # oid ref -> which of the record's paths hashes to this ref (for na return 0 } -commit_plan() { # -> 0 committed; 1 lost a race (refusals printed) - transact && return 0 - local lost=0 ref cur p - for ref in "${PLAN_ORDER[@]}"; do - [[ "${ref}" == "${NS}/paths/"* ]] || continue - cur="$(ref_oid "${ref}")" - [[ -z "${cur}" ]] && continue - p="$(ref_path "${cur}" "${ref}")" - [[ -z "${p}" ]] && continue - describe "${cur}" - in_list "${D_JOB}" "${BATCH_JOBS[@]}" && continue - refusal "${p}" - lost=1 +claim_reset() { # planning state for one attempt at a claim or a batch + plan_reset + BATCH_JOBS=() + BATCH_HOLDER=() + BUMPED=() + BATCH_PATH=() + CONFLICTS=0 +} + +commit_claims() { # plan-fn -> 0 committed; exits 1 refused. plan-fn plans every claim of this command against the current snapshot and sets CONFLICTS + local attempt + for ((attempt = 0; attempt < RETRIES; attempt++)); do + ((attempt > 0)) && snapshot # a lost transaction: read again and plan again, so the refusal names what actually won + claim_reset + "$1" + ((CONFLICTS)) && exit 1 + transact && return 0 + sleep 0.01 done - ((lost == 0)) && transaction_refusal - return 1 + transaction_refusal + exit 1 } claim_args() { # parses claim arguments into CA_JOB CA_HOLDER CA_TTL CA_PARENT CA_NOTE CA_PATHS @@ -241,11 +337,10 @@ claim_args() { # parses claim arguments into CA_JOB CA_HOLDER CA_TTL CA_PARENT C ((${#CA_PATHS[@]} > 0)) || usage } +plan_one_claim() { plan_claim "${CA_JOB}" "${CA_HOLDER}" "${CA_TTL}" "${CA_PARENT}" "${CA_NOTE}" "${CA_PATHS[@]}"; } + cmd_claim() { claim_args "$@" - plan_reset - plan_claim "${CA_JOB}" "${CA_HOLDER}" "${CA_TTL}" "${CA_PARENT}" "${CA_NOTE}" "${CA_PATHS[@]}" - ((CONFLICTS)) && exit 1 - commit_plan || exit 1 + commit_claims plan_one_claim printf '%s\n' "${CLAIM_LINE}" } diff --git a/lib/100-batch.sh b/lib/100-batch.sh index eb0689c..6638899 100644 --- a/lib/100-batch.sh +++ b/lib/100-batch.sh @@ -1,9 +1,32 @@ # ---------------------------------------------------------------- batch +B_JOB=() +B_HOLDER=() +B_TTL=() +B_PARENT=() +B_NOTE=() +B_PATHS=() # newline joined +B_LINES=() + +plan_batch() { # plans every parsed record against the current snapshot; B_LINES collects the claim lines + local i paths p list + B_LINES=() + for i in "${!B_JOB[@]}"; do + list=() + paths="${B_PATHS[${i}]}" + while [[ -n "${paths}" ]]; do + p="${paths%%$'\n'*}" + if [[ "${p}" == "${paths}" ]]; then paths=''; else paths="${paths#*$'\n'}"; fi + [[ -n "${p}" ]] && list+=("${p}") + done + plan_claim "${B_JOB[${i}]}" "${B_HOLDER[${i}]}" "${B_TTL[${i}]}" "${B_PARENT[${i}]}" "${B_NOTE[${i}]}" "${list[@]}" + B_LINES+=("${CLAIM_LINE}") + done +} + cmd_batch() { (($# == 0)) || usage - local line key val job='' holder='' ttl='' parent='' note='' paths=() in_paths=0 count=0 lines_out=() - plan_reset + local line key val job='' holder='' ttl='' parent='' note='' paths=() in_paths=0 count=0 finish_record() { if [[ -z "${job}" && -z "${holder}" && -z "${ttl}" && -z "${parent}" && -z "${note}" && ${#paths[@]} -eq 0 ]]; then return 0; fi # only a wholly empty record is skipped; one with just parent: or ttl: is malformed [[ -n "${job}" && -n "${holder}" && ${#paths[@]} -gt 0 ]] || fail 'batch: every record needs job:, holder: and at least one path under paths:' 2 @@ -11,8 +34,15 @@ cmd_batch() { [[ -z "${ttl}" ]] && ttl="${DEFAULT_TTL}" valid_ttl ttl "${ttl}" || fail 'batch: ttl is a positive number of seconds' 2 valid_holder "${holder}" || fail 'batch: holder must be one line' 2 - plan_claim "${job}" "${holder}" "${ttl}" "${parent}" "${note}" "${paths[@]}" - lines_out+=("${CLAIM_LINE}") + valid_note "${note}" || fail 'batch: note must be one line' 2 + B_JOB+=("${job}") + B_HOLDER+=("${holder}") + B_TTL+=("${ttl}") + B_PARENT+=("${parent}") + B_NOTE+=("${note}") + local joined + joined="$(printf '%s\n' "${paths[@]}")" + B_PATHS+=("${joined}") count=$((count + 1)) job='' holder='' @@ -46,7 +76,6 @@ cmd_batch() { done finish_record ((count > 0)) || fail 'batch: no records on stdin' 2 - ((CONFLICTS)) && exit 1 - commit_plan || exit 1 - printf '%s\n' "${lines_out[@]}" + commit_claims plan_batch + printf '%s\n' "${B_LINES[@]}" } diff --git a/lib/120-check.sh b/lib/120-check.sh index 2388e13..c4cd791 100644 --- a/lib/120-check.sh +++ b/lib/120-check.sh @@ -3,13 +3,21 @@ cmd_check() { ensure_snapshot # in this shell, so the $(…) reads below inherit one fresh snapshot instead of each taking their own (($# > 0)) || usage - local at held=0 p n ref cur jp _j1 _j2 + local at held=0 p n ref cur jp _j1 _j2 via jv now_v at for p in "$@"; do n="$(normalize_path "${p}")" || exit 2 path_ref ref "${n}" cur="$(ref_oid "${ref}")" json_str jp "${n}" + via='' + if [[ -z "${cur}" ]] || ! record_live "${cur}" "${at}"; then + covering_v via "${n}" "${at}" # a live prefix above, or a live path under a prefix; empty when none + fi + if [[ -n "${via}" ]]; then + path_ref ref "${via}" + cur="$(ref_oid "${ref}")" + fi if [[ -z "${cur}" ]]; then printf '{"path":%s,"state":"free"}\n' "${jp}" continue @@ -17,8 +25,13 @@ cmd_check() { describe "${cur}" json_str _j1 "${D_HOLDER}" json_str _j2 "${D_JOB}" + jv='' + if [[ -n "${via}" ]]; then + json_str jv "${via}" + jv=",\"via\":${jv}" + fi if [[ "${D_EXPIRES}" -gt "${at}" ]]; then - printf '{"path":%s,"state":"held","holder":%s%s,"job":%s,"expires":%s,"remaining":%s}\n' "${jp}" "${_j1}" "${D_NOTE_JSON}" "${_j2}" "${D_EXPIRES}" "${D_REMAINING}" + printf '{"path":%s,"state":"held","holder":%s%s,"job":%s%s,"expires":%s,"remaining":%s}\n' "${jp}" "${_j1}" "${D_NOTE_JSON}" "${_j2}" "${jv}" "${D_EXPIRES}" "${D_REMAINING}" held=1 else printf '{"path":%s,"state":"expired","holder":%s%s,"job":%s,"expires":%s,"remaining":0}\n' "${jp}" "${_j1}" "${D_NOTE_JSON}" "${_j2}" "${D_EXPIRES}" @@ -26,3 +39,41 @@ cmd_check() { done return "${held}" } + +record_live() { # oid now -> 0 when the record's expiry is in the future + local _rl + field_v _rl "$1" expires + [[ -n "${_rl}" ]] && ((_rl > $2)) +} + +covering_v() { # VAR path now: set VAR to the path of a live lock that covers this one from above (a prefix) or, for a prefix, from below (a path under it); empty when none + local _cv_p="$2" _cv_at="$3" _cv_ancs _cv_anc _cv_ref _cv_cur _cv_rows _cv_oid _cv_paths _cv_rp + printf -v "$1" '' + ancestors_v _cv_ancs "${_cv_p}" + while [[ -n "${_cv_ancs}" ]]; do + _cv_anc="${_cv_ancs%%$'\n'*}" + if [[ "${_cv_anc}" == "${_cv_ancs}" ]]; then _cv_ancs=''; else _cv_ancs="${_cv_ancs#*$'\n'}"; fi + path_ref _cv_ref "${_cv_anc}" + _cv_cur="${REF_OID[${_cv_ref}]:-}" + if [[ -n "${_cv_cur}" ]] && record_live "${_cv_cur}" "${_cv_at}"; then + printf -v "$1" '%s' "${_cv_anc}" + return 0 + fi + done + is_prefix "${_cv_p}" || return 0 + _cv_rows="$(job_refs)" + while IFS=' ' read -r _cv_ref _cv_oid; do + [[ -z "${_cv_ref}" ]] && continue + record_live "${_cv_oid}" "${_cv_at}" || continue + record_paths_v _cv_paths "${_cv_oid}" + while [[ -n "${_cv_paths}" ]]; do + _cv_rp="${_cv_paths%%$'\n'*}" + if [[ "${_cv_rp}" == "${_cv_paths}" ]]; then _cv_paths=''; else _cv_paths="${_cv_paths#*$'\n'}"; fi + if [[ -n "${_cv_rp}" && "${_cv_rp}" == "${_cv_p}"?* ]]; then + printf -v "$1" '%s' "${_cv_rp}" + return 0 + fi + done + done <<<"${_cv_rows}" + return 0 +} diff --git a/lib/175-doctor.sh b/lib/175-doctor.sh index 74cf545..54b8dac 100644 --- a/lib/175-doctor.sh +++ b/lib/175-doctor.sh @@ -146,6 +146,7 @@ cmd_doctor() { PATHREF_OID["${ref}"]="${oid}" pathrefs+=("${ref}") ;; + "${NS}"/dirs/*) ;; # a directory token: the last record that touched the directory; any object will do, the snapshot already checked it exists "${NS}"/sem/*) rest="${ref#"${NS}"/sem/}" name="${rest%%/*}" diff --git a/schema/git-locks.schema.json b/schema/git-locks.schema.json index 1426d18..97f8e1a 100644 --- a/schema/git-locks.schema.json +++ b/schema/git-locks.schema.json @@ -70,7 +70,7 @@ "path": { "type": "string", "minLength": 1, - "description": "A repo-relative path, normalised: no leading ./, never absolute, never containing a .. component or a newline." + "description": "A repo-relative path, normalised: no leading ./, never absolute, never containing a .. component or a newline. A trailing / makes it a prefix: dist/ covers every path under dist/ (and is covered by any lock under it); dist without the slash is the directory entry itself, a different key." }, "epoch": { "type": "integer", @@ -104,6 +104,9 @@ "job": { "$ref": "#/$defs/job" }, + "via": { + "$ref": "#/$defs/via" + }, "expires": { "$ref": "#/$defs/epoch" }, @@ -150,6 +153,11 @@ "required": [ "note" ] + }, + { + "required": [ + "via" + ] } ] } @@ -304,6 +312,9 @@ "job": { "$ref": "#/$defs/job" }, + "via": { + "$ref": "#/$defs/via" + }, "expires": { "$ref": "#/$defs/epoch" } @@ -376,6 +387,9 @@ }, "path": { "$ref": "#/$defs/path" + }, + "via": { + "$ref": "#/$defs/via" } }, "additionalProperties": false @@ -1066,6 +1080,10 @@ "minLength": 1, "pattern": "^[^\\n\\r]*$", "description": "One line saying why the lock is held, given at claim time with --note (or note: in a batch record); absent when none was given. Carried on every line that names the lock." + }, + "via": { + "$ref": "#/$defs/path", + "description": "The covering lock's own path when it differs from the one asked about or claimed: the prefix above it, or the path under a wanted prefix. Absent when the lock is on the path itself." } } } diff --git a/test/test.sh b/test/test.sh index d157301..22657a5 100755 --- a/test/test.sh +++ b/test/test.sh @@ -107,11 +107,36 @@ jstr() { # VAR JSON-LINE KEY: the string value of KEY (first occurrence), or emp printf -v "$1" '%s' "${val}" } -refs() { # subject-repo [prefix] -> refs in whatever store resolves for it +OUT_WAITER="$(mktemp "${TMPDIR:-/tmp}/git-locks-out.XXXXXX")" +OUT_WAITER2="$(mktemp "${TMPDIR:-/tmp}/git-locks-out.XXXXXX")" +OUT_SWEEP="$(mktemp "${TMPDIR:-/tmp}/git-locks-out.XXXXXX")" +OUT_PRE="$(mktemp "${TMPDIR:-/tmp}/git-locks-out.XXXXXX")" +ERR_PRE="$(mktemp "${TMPDIR:-/tmp}/git-locks-out.XXXXXX")" +OUT_LEAF3="$(mktemp "${TMPDIR:-/tmp}/git-locks-out.XXXXXX")" +ERR_LEAF3="$(mktemp "${TMPDIR:-/tmp}/git-locks-out.XXXXXX")" + +reached() { # gate-path: wait until the paused command signals it has reached that gate (test_gate writes .ready before waiting) + local waited=0 + until [[ -e "$1.ready" ]] || ((waited >= 600)); do + sleep 0.05 + waited=$((waited + 1)) + done + [[ -e "$1.ready" ]] || printf ' FAIL the paused command never reached %s\n' "$1" +} + +refs() { # subject-repo [prefix] -> refs in whatever store resolves for it; directory tokens (refs/locks/dirs/, not locks) are left out unless asked for + local store line + line="$(cd "$1" && git-locks store)" || return 1 + jstr store "${line}" store + git --git-dir="${store}" for-each-ref --format='%(refname)' "refs/locks/${2:-}" | grep -v '^refs/locks/dirs/' | sort + return 0 +} + +dir_tokens() { # subject-repo -> the number of directory token refs local store line line="$(cd "$1" && git-locks store)" || return 1 jstr store "${line}" store - git --git-dir="${store}" for-each-ref --format='%(refname)' "refs/locks/${2:-}" | sort + git --git-dir="${store}" for-each-ref --format='%(refname)' 'refs/locks/dirs/' | wc -l | tr -d ' ' } export GIT_LOCKS_NOW=1000000 @@ -831,7 +856,7 @@ git-locks claim --job P --holder h p.md >/dev/null 2>&1 GATE="$(mktemp -d "${TMPDIR:-/tmp}/git-locks-gate.XXXXXX")/go" GIT_LOCKS_PAUSE_BEFORE_COMMIT="${GATE}" git-locks release --job P >/dev/null 2>&1 & rel=$! -sleep 1 # the release has read the family (no children) and is paused before its transaction +reached "${GATE}" # the release has read the family (no children) and is paused before its transaction git-locks claim --job C --parent P --holder h c.md >/dev/null 2>&1 check "a child claim while a release is paused before commit succeeds (the parent still exists)" "$?" "0" : >"${GATE}" @@ -915,7 +940,7 @@ check "dir//file.md names the same path as dir/file.md" "$?" "1" git-locks check 'dir/./file.md' >/dev/null 2>&1 check "dir/./file.md names the same path as dir/file.md" "$?" "1" git-locks check 'dir/file.md/' >/dev/null 2>&1 -check "a trailing slash is stripped" "$?" "1" +check "a trailing slash is a prefix, not the same key: dir/file.md/ asks about paths under dir/file.md, and none is held" "$?" "0" # ---------------------------------------------------------------- acquisition identity survives renewal (review MUST 4, second half) @@ -976,7 +1001,7 @@ git-locks claim --job P --holder h p.md >/dev/null 2>&1 GATE="$(mktemp -d "${TMPDIR:-/tmp}/git-locks-gate2.XXXXXX")/go" GIT_LOCKS_PAUSE_BEFORE_COMMIT="${GATE}" git-locks release --job P >/dev/null 2>&1 & rel=$! -sleep 1 +reached "${GATE}" git-locks extend --job P --ttl 4242 >/dev/null 2>&1 check "a renewal while a release is paused before commit succeeds" "$?" "0" : >"${GATE}" @@ -994,14 +1019,14 @@ git-locks sem acquire w3 --job holder --holder o >/dev/null 2>&1 GATE3="$(mktemp -d "${TMPDIR:-/tmp}/git-locks-gate3.XXXXXX")/go" TRACE3="$(mktemp "${TMPDIR:-/tmp}/git-locks-trace3.XXXXXX")" # The waiter reads (semaphore full), then pauses after that read. The holder releases. The gate opens. -GIT_LOCKS_PAUSE_AFTER_READ="${GATE3}" GIT_LOCKS_TRACE="${TRACE3}" git-locks sem acquire w3 --job waiter --holder h --wait 20 >/tmp/gl-waiter.out 2>&1 & +GIT_LOCKS_PAUSE_AFTER_READ="${GATE3}" GIT_LOCKS_TRACE="${TRACE3}" git-locks sem acquire w3 --job waiter --holder h --wait 20 >"${OUT_WAITER}" 2>&1 & wpid=$! -sleep 1 +reached "${GATE3}" git-locks sem release w3 --job holder >/dev/null 2>&1 : >"${GATE3}" wait "${wpid}" check "the waiter acquires after the release it could not see at first" "$?" "0" -out="$(cat /tmp/gl-waiter.out)" +out="$(cat "${OUT_WAITER}")" jfields "the waiter's line is a real acquisition" "${out}" 'event="acquired"' 'job="waiter"' 'live=1' 'capacity=1' reads="$(grep -c '^snapshot' "${TRACE3}")" check "the waiter took exactly two reads: the stale one it was paused on, and one fresh read that saw the release" "${reads}" "2" @@ -1012,14 +1037,14 @@ jfields "the semaphore holds one live slot, the waiter's" "${out}" 'live=1' 'cap git-locks claim --job blocker --holder o p3.md >/dev/null 2>&1 GATE4="$(mktemp -d "${TMPDIR:-/tmp}/git-locks-gate4.XXXXXX")/go" TRACE4="$(mktemp "${TMPDIR:-/tmp}/git-locks-trace4.XXXXXX")" -GIT_LOCKS_PAUSE_AFTER_READ="${GATE4}" GIT_LOCKS_TRACE="${TRACE4}" git-locks with --job waiter2 --holder h --wait 20 p3.md -- sh -c "cp '${TRACE4}' '${TRACE4}.at-run'; echo ran" >/tmp/gl-waiter2.out 2>/dev/null & +GIT_LOCKS_PAUSE_AFTER_READ="${GATE4}" GIT_LOCKS_TRACE="${TRACE4}" git-locks with --job waiter2 --holder h --wait 20 p3.md -- sh -c "cp '${TRACE4}' '${TRACE4}.at-run'; echo ran" >"${OUT_WAITER2}" 2>/dev/null & wpid=$! -sleep 1 +reached "${GATE4}" git-locks release --job blocker >/dev/null 2>&1 : >"${GATE4}" wait "${wpid}" check "with --wait runs its command after a release it could not see at first" "$?" "0" -ran="$(cat /tmp/gl-waiter2.out)" +ran="$(cat "${OUT_WAITER2}")" check "and the command ran once" "${ran}" "ran" reads="$(grep -c '^snapshot' "${TRACE4}.at-run")" check "with took exactly two reads before running its command (the release afterwards is a third)" "${reads}" "2" @@ -1304,14 +1329,14 @@ R="$(mkrepo)" cd "${R}" || exit 2 GIT_LOCKS_NOW=1000 git-locks claim --job renew --holder h --ttl 10 r.md >/dev/null 2>&1 GATE6="$(mktemp -d "${TMPDIR:-/tmp}/git-locks-gate6.XXXXXX")/go" -GIT_LOCKS_NOW=2000 GIT_LOCKS_PAUSE_AFTER_READ="${GATE6}" git-locks sweep >/tmp/gl-sweep.out 2>&1 & +GIT_LOCKS_NOW=2000 GIT_LOCKS_PAUSE_AFTER_READ="${GATE6}" git-locks sweep >"${OUT_SWEEP}" 2>&1 & spid=$! -sleep 1 +reached "${GATE6}" GIT_LOCKS_NOW=2000 git-locks extend --job renew --ttl 100 >/dev/null 2>&1 : >"${GATE6}" wait "${spid}" check "sweep exits 0 when the expired lock it saw was renewed underneath" "$?" "0" -out="$(cat /tmp/gl-sweep.out)" +out="$(cat "${OUT_SWEEP}")" check "and sweeps nothing" "${out}" "" out="$(GIT_LOCKS_NOW=2000 git-locks ttl --job renew 2>&1)" jfields "the renewed lock is still there with its new expiry" "${out}" 'expires=2100' @@ -1378,6 +1403,174 @@ out="$(git-locks claim --job q --holder alice --note 'say "hi"' q.md 2>&1)" jfields "a note is JSON-escaped" "${out}" 'note="say \"hi\""' valid "claim line with a quoted note" "${out}" +# ---------------------------------------------------------------- #6: prefix locks + +R="$(mkrepo)" +cd "${R}" || exit 2 +out="$(git-locks claim --job build --holder alice dist/ 2>&1)" +check "a claim on dist/ exits 0" "$?" "0" +jfields "and the claim line keeps the trailing slash: the path is a prefix" "${out}" 'paths=["dist/"]' +err="$(git-locks claim --job other --holder bob dist/a.js 2>&1 >/dev/null)" +check "a claim on dist/a.js by another job is refused: dist/ covers it" "$?" "1" +jfields "the refusal names the holder, the job and the prefix that covers the path" "${err}" 'path="dist/a.js"' 'holder="alice"' 'job="build"' 'via="dist/"' +valid "refusal line with via" "${err}" +git-locks show --job other >/dev/null 2>&1 +check "and nothing landed for the loser" "$?" "1" +out="$(git-locks check dist/a.js 2>&1)" +check "check dist/a.js is held" "$?" "1" +jfields "and says which prefix holds it" "${out}" 'state="held"' 'job="build"' 'via="dist/"' +valid "check line with via" "${out}" +out="$(git-locks check dist/ 2>&1)" +check "check dist/ is held, by the prefix lock itself" "$?" "1" +rc=0 +[[ "${out}" != *'"via"'* ]] || rc=1 +check "with no via: the path asked about is the lock's own" "${rc}" "0" +git-locks check dist >/dev/null 2>&1 +check "check dist (no slash) is free: a prefix covers what is under it, not the directory entry itself" "$?" "0" +git-locks check other/a.js >/dev/null 2>&1 +check "a path elsewhere is free" "$?" "0" +out="$(git-locks claim --job build --holder alice dist/ dist/a.js 2>&1)" +check "the same job may claim under its own prefix" "$?" "0" +git-locks release --job build >/dev/null 2>&1 + +# The other direction: a prefix claim over a held path. +git-locks claim --job a --holder alice src/x.md >/dev/null 2>&1 +err="$(git-locks claim --job b --holder bob src/ 2>&1 >/dev/null)" +check "a claim on src/ while src/x.md is held by another job is refused" "$?" "1" +jfields "the refusal names the held path under the prefix" "${err}" 'path="src/"' 'job="a"' 'via="src/x.md"' +out="$(git-locks check src/ 2>&1)" +check "check src/ is held" "$?" "1" +jfields "via the held path under it" "${out}" 'via="src/x.md"' +git-locks claim --job c --holder carol s/ >/dev/null 2>&1 +check "s/ is not a prefix of src/x.md: a sibling prefix claims" "$?" "0" +git-locks claim --job d --holder dan srcs/y.md >/dev/null 2>&1 +check "srcs/y.md is not under src/: it claims" "$?" "0" +git-locks release --job a --job c --job d >/dev/null 2>&1 + +# Nested prefixes conflict both ways, across jobs. +git-locks claim --job outer --holder alice a/ >/dev/null 2>&1 +err="$(git-locks claim --job inner --holder bob a/b/ 2>&1 >/dev/null)" +check "a/b/ under a held a/ is refused" "$?" "1" +jfields "via a/" "${err}" 'via="a/"' +git-locks release --job outer >/dev/null 2>&1 +git-locks claim --job inner --holder bob a/b/ >/dev/null 2>&1 +err="$(git-locks claim --job outer --holder alice a/ 2>&1 >/dev/null)" +check "a/ over a held a/b/ is refused" "$?" "1" +jfields "via a/b/" "${err}" 'via="a/b/"' +git-locks release --job inner >/dev/null 2>&1 + +# Expired locks in the way are evicted, in either direction. +GIT_LOCKS_NOW=1000 git-locks claim --job old --holder alice --ttl 10 src/y.md >/dev/null 2>&1 +GIT_LOCKS_NOW=2000 git-locks claim --job sweeper --holder bob src/ >/dev/null 2>&1 +check "a prefix claim over an expired lock under it succeeds" "$?" "0" +GIT_LOCKS_NOW=2000 git-locks show --job old >/dev/null 2>&1 +check "and the expired lock was evicted" "$?" "1" +GIT_LOCKS_NOW=2000 git-locks release --job sweeper >/dev/null 2>&1 +GIT_LOCKS_NOW=1000 git-locks claim --job oldp --holder alice --ttl 10 lib/ >/dev/null 2>&1 +GIT_LOCKS_NOW=2000 git-locks claim --job leaf --holder bob lib/z.md >/dev/null 2>&1 +check "a claim under an expired prefix succeeds" "$?" "0" +GIT_LOCKS_NOW=2000 git-locks show --job oldp >/dev/null 2>&1 +check "and the expired prefix was evicted" "$?" "1" +GIT_LOCKS_NOW=2000 git-locks release --job leaf >/dev/null 2>&1 + +# Normalisation keeps the marker and nothing else. +git-locks claim --job n --holder alice 'dir//' >/dev/null 2>&1 +git-locks check 'dir/' >/dev/null 2>&1 +check "dir// is the prefix dir/" "$?" "1" +git-locks check './dir/./' >/dev/null 2>&1 +check "./dir/./ is the prefix dir/" "$?" "1" +out="$(git-locks list 2>&1)" +jfields "list shows the prefix with its slash" "${out}" 'paths=["dir/"]' +git-locks release --job n >/dev/null 2>&1 +out="$(git-locks claim --job root --holder alice '/' 2>&1)" +check "a bare slash is an empty path, refused" "$?" "2" + +# with, the case in the issue: with --job build dist/ -- make protects everything under dist/. +out="$(git-locks with --job build --holder alice dist/ -- git-locks check dist/a.js 2>/dev/null)" +check "under with --job build dist/, dist/a.js is held" "$?" "1" +jfields "via dist/" "${out}" 'via="dist/"' +git-locks check dist/a.js >/dev/null 2>&1 +check "and free once with returns" "$?" "0" + +# A batch of two claims under one directory moves that directory's token once. +out="$(printf 'job: b1\nholder: alice\npaths:\nd/one.md\n\njob: b2\nholder: alice\npaths:\nd/two.md\n' | git-locks batch 2>&1)" +check "a batch of two claims in one directory succeeds" "$?" "0" +lines n "${out}" +check "with two claim lines" "${n}" "2" +git-locks release --job b1 --job b2 >/dev/null 2>&1 + +# The race, forced both ways. First: a prefix claim reads, pauses before commit; a path under it lands meanwhile. +GATE7="$(mktemp -d "${TMPDIR:-/tmp}/git-locks-gate7.XXXXXX")/go" +GIT_LOCKS_PAUSE_BEFORE_COMMIT="${GATE7}" git-locks claim --job pre --holder alice r/ >"${OUT_PRE}" 2>"${ERR_PRE}" & +ppid=$! +reached "${GATE7}" # it has read, planned and is paused before its transaction; only then does the competitor run +git-locks claim --job leaf2 --holder bob r/f.md >/dev/null 2>&1 +check "the path claim lands while the prefix claim is paused" "$?" "0" +: >"${GATE7}" +wait "${ppid}" +check "the paused prefix claim is refused: its transaction failed and the re-plan saw the path" "$?" "1" +err="$(cat "${ERR_PRE}")" +jfields "and it says via which path" "${err}" 'via="r/f.md"' +git-locks show --job pre >/dev/null 2>&1 +check "no prefix lock landed" "$?" "1" +git-locks release --job leaf2 >/dev/null 2>&1 +# Second: a path claim pauses before commit; a prefix over it lands meanwhile. +GATE8="$(mktemp -d "${TMPDIR:-/tmp}/git-locks-gate8.XXXXXX")/go" +GIT_LOCKS_PAUSE_BEFORE_COMMIT="${GATE8}" git-locks claim --job leaf3 --holder bob r2/f.md >"${OUT_LEAF3}" 2>"${ERR_LEAF3}" & +ppid=$! +reached "${GATE8}" +git-locks claim --job pre2 --holder alice r2/ >/dev/null 2>&1 +check "the prefix claim lands while the path claim is paused" "$?" "0" +: >"${GATE8}" +wait "${ppid}" +check "the paused path claim is refused: the prefix landed first" "$?" "1" +err="$(cat "${ERR_LEAF3}")" +jfields "via the prefix" "${err}" 'via="r2/"' +git-locks release --job pre2 >/dev/null 2>&1 + +out="$(git-locks doctor 2>&1)" +check "doctor is healthy with prefix locks and directory tokens in the store" "$?" "0" + +# Directory tokens are bookkeeping, not locks: one per directory level ever touched, left in place by release. +R="$(mkrepo)" +cd "${R}" || exit 2 +git-locks claim --job t --holder alice a/b/c.md >/dev/null 2>&1 +n="$(dir_tokens "${R}")" +check "a claim two directories deep leaves two directory tokens (a/ and a/b/)" "${n}" "2" +git-locks claim --job t2 --holder bob a/b/d.md >/dev/null 2>&1 +n="$(dir_tokens "${R}")" +check "a second claim in the same directories adds none" "${n}" "2" +git-locks release --job t --job t2 >/dev/null 2>&1 +n="$(dir_tokens "${R}")" +check "release leaves them: they record the last claim that touched the directory, not a lock" "${n}" "2" +git-locks check a/b/c.md >/dev/null 2>&1 +check "and the paths are free" "$?" "0" + +# ---------------------------------------------------------------- a batch cannot straddle a prefix boundary across jobs + +R="$(mkrepo)" +cd "${R}" || exit 2 +out="$(printf 'job: bp1\nholder: alice\npaths:\ndist/\n\njob: bp2\nholder: bob\npaths:\ndist/a.js\n' | git-locks batch 2>&1 >/dev/null)" +check "a batch claiming dist/ for one job and dist/a.js for another is refused" "$?" "1" +jfields "as a duplicate, naming the path and the record that covers it" "${out}" 'reason="duplicate"' 'path="dist/a.js"' 'via="dist/"' +valid "the batch overlap refusal" "${out}" +got="$(refs "${R}")" +check "and nothing at all landed" "${got}" "" +out="$(printf 'job: bp1\nholder: alice\npaths:\ndist/a.js\n\njob: bp2\nholder: bob\npaths:\ndist/\n' | git-locks batch 2>&1 >/dev/null)" +check "the other order is refused too" "$?" "1" +jfields "naming the prefix and the path it covers" "${out}" 'reason="duplicate"' 'path="dist/"' 'via="dist/a.js"' +got="$(refs "${R}")" +check "and nothing landed then either" "${got}" "" +out="$(printf 'job: deep\nholder: alice\npaths:\na/b/\n\njob: deeper\nholder: bob\npaths:\na/b/c/d.md\n' | git-locks batch 2>&1 >/dev/null)" +check "two levels down is still an overlap" "$?" "1" +out="$(printf 'job: same\nholder: alice\npaths:\ndist/\ndist/a.js\n' | git-locks batch 2>&1)" +check "one job may hold a prefix and a path under it in one batch" "$?" "0" +git-locks release --job same >/dev/null 2>&1 +out="$(printf 'job: s1\nholder: alice\npaths:\nsrc/\n\njob: s2\nholder: bob\npaths:\nsrcs/\n' | git-locks batch 2>&1)" +check "sibling prefixes in one batch are not an overlap" "$?" "0" +lines n "${out}" +check "and both records claimed" "${n}" "2" + printf '\n%d passed, %d failed\n' "${PASS}" "${FAIL}" if ((FAIL > 0)); then printf 'failed: %s\n' "${FAILED[@]}"