Broken pipe/worker connection closed errors in TUI only, inconsistently

[anova-andy]

Environment

• sqlit-tui 1.4.0
• Python 3.14.4 (same with 3.12 and 3.13)
• MacOS Tahoe 26.4.1
• psycopg2-binary (injected with pipx)

Issue

CLI query of a Postgres database such as sqlit -c 'conn' -q 'select * from abcd' works fine and returns results, every time. In TUI mode, I'm able to establish a connection, but can't browse tables or run queries as I get the following errors:

(browse)

(query)

If I disconnect and reconnect within the TUI, I can browse and query normally, but this only works maybe 1/5 times.

[brentrager]

Hit this on Python 3.12.13 (so the 3.12 downgrade workaround from #149 doesn't help). Got a worker-side traceback by patching run_process_worker to redirect sys.stderr + faulthandler to a file. The "Worker connection closed." error is a downstream symptom of a segfault in psycopg2.connect() caused by the spawn → fork fallback on macOS.

=== worker pid=96186 started ===
Fatal Python error: Segmentation fault

Current thread (most recent call first):
  File ".../psycopg2/__init__.py", line 122 in connect
  File ".../sqlit/domains/connections/providers/postgresql/adapter.py", line 78 in connect
  File ".../sqlit/domains/process_worker/app/process_worker.py", line 396 in run
  ...

Thread (most recent call first):
  File ".../multiprocessing/popen_fork.py", line 71 in _launch
  File ".../sqlit/domains/process_worker/app/process_worker_client.py", line 56 in _start_with_context
  File ".../sqlit/domains/process_worker/app/process_worker_client.py", line 67 in _maybe_fallback_start
  ...

Extension modules: ..., psycopg2._psycopg

What's happening

1. ProcessWorkerClient.__init__ tries get_context("spawn") first.
2. spawn fails with ValueError: bad value(s) in fds_to_keep (likely a Textual-held FD that isn't inheritable on macOS+3.12).
3. _maybe_fallback_start falls back to get_context("fork").
4. Forking after libpq / Security.framework is loaded is unsafe on macOS — psycopg2 segfaults in its C extension when the forked child calls connect().
5. Pipe dies → parent surfaces the generic Worker connection closed. message.

I also tried forkserver as the fallback — same fds_to_keep error before the worker entrypoint runs, so not viable either.

Repro

• macOS Tahoe 26.4.1, Python 3.12.13, sqlit-tui 1.4.0 (pipx), psycopg2-binary
• PostgreSQL/RDS over TLS
• Expand Tables/Views or run any query — intermittent (spawn occasionally succeeds)

Mitigation

Don't fall back to fork on darwin — let the in-process fallback in query_execution.py:486 handle it. In process_worker_client.py:

def _maybe_fallback_start(self, error: Exception) -> None:
    if sys.platform == "darwin":
        raise error
    # ... existing logic

Popup becomes Process worker unavailable; falling back. (bad value(s) in fds_to_keep), queries run in-process, no crash. Only loss is mid-query cancellation.

Real fix (root cause)

The fds_to_keep failure on macOS+Textual+Python 3.12 needs to be addressed for the process worker to actually work — likely pre-spawning the worker before Textual takes over, or switching to psycopg3 + asyncio.

[brentrager]

Addendum — tested fixes and what actually works on macOS Tahoe

After more debugging on macOS Tahoe (26.4.1) + Apple Silicon I want to extend the earlier comment with what we tested. The short version: swapping to psycopg3 alone is not enough — the fork fallback itself is fundamentally broken on Tahoe and needs to be removed.

What we tried

Mitigation	Outcome
Pin psycopg2-binary==2.9.5 (avoids the libssl bundling regression in 2.9.6+)	Still crashes — the fork is the real problem, not psycopg2's wheel
OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES	Still crashes — doesn't cover Tahoe's expanded crash surface
Source-built psycopg2 against system libpq	Still crashes
Switch driver to psycopg[binary] (psycopg3)	Crash moves — no longer in psycopg.connect, now in getaddrinfo
Disable fork fallback on darwin	Fixes it — in-process fallback runs cleanly
Disable fork fallback + switch to psycopg3	Best result — when spawn succeeds, full worker works perfectly

Second crash site we observed (with psycopg3)

After swapping to psycopg3, the spawn still occasionally fails with fds_to_keep, fork is attempted, and the child segfaults at a different location:

Fatal Python error: Segmentation fault

Current thread [Thread-1 (run)] (most recent call first):
  File ".../socket.py", line 983 in getaddrinfo
  File ".../psycopg/_conninfo_attempts.py", line 102 in _resolve_hostnames
  File ".../psycopg/connection.py", line 100 in connect
  File ".../sqlit/.../postgresql/adapter.py", line 78 in connect
  ...

C stack:
  libsystem_trace.dylib `_os_log_find
  libsystem_trace.dylib `os_log_create
  libsystem_info.dylib `gai_log_init
  libsystem_info.dylib `_gai_nat64_second_pass
  libsystem_info.dylib `getaddrinfo

This is the same root cause family as celery #9894 (setproctitle on Tahoe) and gunicorn #3526 (NSCharacterSet on Tahoe): macOS Tahoe widened the set of system libraries that abort when initialized after fork(). os_log, libsystem_info, NSCharacterSet, libsystem_trace, and Security.framework all now hit the abort path. The fork itself is the bug; it doesn't matter what Python calls inside the child.

Why "intermittent"

Spawn fails non-deterministically because Textual's event loop holds non-inheritable FDs that come and go during normal operation. Sqlit's worker is created lazily from idle_scheduler.py:_warm, so the spawn timing is racy. Sometimes Textual is "clean" and spawn succeeds; sometimes it isn't and spawn raises fds_to_keep, then fork is attempted, then crash.

Recommended patch (minimal)

In process_worker_client.py:

def _maybe_fallback_start(self, error: Exception) -> None:
    # Don't fall back to fork on macOS (Tahoe+): forking after the parent
    # has loaded CoreFoundation/Security.framework/os_log causes segfaults
    # in the child (e.g. inside psycopg2 TLS init or libsystem_trace's
    # gai_log_init). Let the in-process fallback in query_execution handle
    # it instead.
    if sys.platform == "darwin":
        raise error
    # ... existing logic

This costs query cancellation when spawn fails, but eliminates the crashes entirely. The popup Process worker unavailable; falling back. (bad value(s) in fds_to_keep) is informative enough.

Recommended bonus fixes

1. Switch the postgres adapter to psycopg3. psycopg3-binary delegates SSL to libpq instead of bundling libssl, so it doesn't pull Security.framework into the parent at import time. Even when spawn does succeed, this avoids one specific crash class. The only adapter changes needed are import psycopg as psycopg2 and database= → dbname=.

2. Redirect the worker's stdout/stderr in run_process_worker. The worker can SIGPIPE on libpq notice writes when its inherited FDs point to a Textual-managed pipe. A 4-line redirect at the top of the worker entrypoint (plus faulthandler.enable(file=...)) avoids that and gives diagnostic visibility for any future crashes:

def run_process_worker(conn):
    import sys, faulthandler
    _log = open("/tmp/sqlit-worker.log", "a", buffering=1)
    sys.stderr = sys.stdout = _log
    faulthandler.enable(file=_log)
    # ... existing body

3. Long-term: pre-spawn the worker before Textual takes over. That eliminates the fds_to_keep race entirely and lets the worker actually run on macOS without falling back. The current lazy creation from the idle scheduler is the source of the timing race.

Related issues confirming Tahoe is the wider regression

• gunicorn macOS: Workers crash with NSCharacterSet fork-safety error after upgrading to 25.1.0 benoitc/gunicorn#3526 (open) — NSCharacterSet fork-safety abort on Tahoe + gunicorn 25.1.0
• celery Crash in setproctitle on macOS Tahoe (26.0) #9893 celery/celery#9894 (open, milestoned 5.7.0) — setproctitle crash on Tahoe via libsystem_trace
• psycopg2 psycopg2-binary 2.9.6 breaks celery worker on Apple silicon psycopg/psycopg2#1593 (closed) — libssl bundling pulls in Security.framework on macOS; psycopg3 fixes
• cpython multiprocessing's default posix start method of 'fork' is broken: change to `'forkserver' || 'spawn' python/cpython#84559 (resolved) — CPython 3.14 makes forkserver the default on POSIX, partial mitigation