Skip to content

Stop createTables from hanging on table locks - #369

Merged
sjoelund merged 1 commit into
masterfrom
createtables-lock-hang
Oct 6, 2026
Merged

sjoelund merged 1 commit into
masterfrom
createtables-lock-hang

Conversation

@sjoelund

@sjoelund sjoelund commented Oct 6, 2026

Copy link
Copy Markdown
Member

Jobs got stuck at 0% CPU right after fetching the reference files. On the shared PostgreSQL database every run kept the transaction of its first SELECT open until the results were written, holding a share lock on libversion, omcversion and the branch table for hours. The next run's createTables issued ALTER TABLE ... ADD COLUMN IF NOT EXISTS and CREATE INDEX IF NOT EXISTS, which take their table lock before checking whether there is anything to do, so it waited for those runs to finish - and every later reader of the table queued behind it.

  • createTables only alters or indexes what is actually missing, and sets a lock_timeout so a needed lock fails the run instead of hanging it.
  • test.py commits after choosing the libraries and after querying the expected execution times, so a run no longer locks the tables while testing.
  • Run test.py with PYTHONUNBUFFERED: stdbuf does not affect Python, so its output lagged behind git's and a hang looked like it was in git. Also print each library before loading it.
  • NeuralNetwork and URDFModelica reference files live on main; the reset to the default origin/master failed and left them stale.

Assisted-by: Claude Opus 5.5

Jobs got stuck at 0% CPU right after fetching the reference files.
On the shared PostgreSQL database every run kept the transaction of
its first SELECT open until the results were written, holding a share
lock on libversion, omcversion and the branch table for hours. The next
run's createTables issued `ALTER TABLE ... ADD COLUMN IF NOT EXISTS`
and `CREATE INDEX IF NOT EXISTS`, which take their table lock before
checking whether there is anything to do, so it waited for those runs
to finish - and every later reader of the table queued behind it.

- createTables only alters or indexes what is actually missing, and
  sets a lock_timeout so a needed lock fails the run instead of
  hanging it.
- test.py commits after choosing the libraries and after querying the
  expected execution times, so a run no longer locks the tables while
  testing.
- Run test.py with PYTHONUNBUFFERED: stdbuf does not affect Python, so
  its output lagged behind git's and a hang looked like it was in git.
  Also print each library before loading it.
- NeuralNetwork and URDFModelica reference files live on `main`; the
  reset to the default `origin/master` failed and left them stale.

Assisted-by: Claude Opus 5.5
@sjoelund
sjoelund enabled auto-merge (squash) October 6, 2026 13:05
@sjoelund
sjoelund disabled auto-merge October 6, 2026 13:05
@sjoelund
sjoelund merged commit 626032b into master Oct 6, 2026
7 checks passed
@sjoelund
sjoelund deleted the createtables-lock-hang branch October 6, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant