_rebuild_lock unlinks .rebuild.lock on release, so two rebuilds can hold the lock at once (POSIX) #4273

Version: graphifyy 0.9.82 (same code in 0.8.40) · macOS (Darwin 25.2) · Python 3.11

What happens

graphify/watch.py _rebuild_lock opens graphify-out/.rebuild.lock (L200), takes fcntl.flock (L209), and on release calls LOCK_UN (L244) and then unlinks the path (L254-258).

flock locks the open file (the inode), not the path. Once the path is unlinked while another process is already waiting on the old inode, a later caller creates a new inode and locks it immediately:

1. A holds the lock.
2. B, a blocking waiter (graphify update → _rebuild_code(..., block_on_lock=True), cli.py:2563), opens the path → inode X and waits in flock.
3. A finishes: LOCK_UN, then unlink(path). B now holds the lock on X, which no longer has a path.
4. C (another graphify update, a post-commit/post-checkout hook, or watch) opens the path → new inode Y → flock succeeds at once.

B and C are now both inside _rebuild_code and write the same outputs (.graph.tmp.json, graph.json, the report). This only needs one blocking waiter plus any later caller, so two terminals/agents running graphify update on the same repo are enough.

Repro (no git repo needed)

import multiprocessing as mp, tempfile, time
from pathlib import Path
from graphify.watch import _rebuild_lock

def holder(out, name, hold, q):
    with _rebuild_lock(out, blocking=True) as got:
        q.put((name, "enter", time.monotonic()))
        time.sleep(hold)
        q.put((name, "exit", time.monotonic()))

if __name__ == "__main__":
    out = Path(tempfile.mkdtemp()) / "graphify-out"
    q = mp.Queue()
    a = mp.Process(target=holder, args=(out, "A", 1.0, q)); a.start(); time.sleep(0.3)
    b = mp.Process(target=holder, args=(out, "B", 1.5, q)); b.start()   # waits on inode X
    a.join(); time.sleep(0.3)                                            # A unlocks + unlinks; B acquires X
    c = mp.Process(target=holder, args=(out, "C", 0.5, q)); c.start()   # new inode Y
    b.join(); c.join()
    ev = sorted((q.get() for _ in range(6)), key=lambda e: e[2])
    for n, k, t in ev:
        print(f"{t - ev[0][2]:5.2f}s {n} {k}")

Output on 0.9.82:

 0.00s A enter
 1.00s A exit
 1.00s B enter
 1.38s C enter      <- C enters while B is still holding the lock
 1.88s C exit
 2.50s B exit

Expected

At most one holder of the rebuild lock at a time.

Possible fixes

• Keep the lock file on release (rewrite/truncate the PID line instead of unlinking). Tools that currently poll for the file's absence could try a non-blocking flock, or read an explicit "done" marker.
• Or keep unlink-on-release but make acquisition robust: after flock succeeds, compare os.fstat(fh.fileno()).st_ino with os.stat(lock_path).st_ino and retry when they differ (or when the path is gone).

Related: #858 (unlink was added there for the stale-PID problem), #1037, #1059, #3881.

Workaround we use for now: serialize every call through a lock file that is never removed, e.g. /usr/bin/lockf -k .git/graphify-update.lock graphify update . on macOS (flock from util-linux on Linux).