Large session loads stall in quadratic JSON-RPC line buffering

[zhuwenzhuang]

Problem

createJSONRPCReader() in src/StdUtils.ts repeatedly concatenates and scans the entire unfinished JSON line for every stdout chunk. A large thread/turns/list response can therefore spend most of a session-load timeout in the transport reader, before ACP history replay begins.

Observed with codex-acp 1.12.0 and Codex CLI 0.155.1 on Linux; the same reader is still present on current main (70273106a0a51839793d58b32645cdd1f44eeca7, package version 1.13.0).

Diagnosis

The reader currently does:

buf += chunk.toString();
const i = buf.indexOf('\n');

Until the final newline arrives, indexOf starts at zero on every chunk. Repeated string concatenation/flattening and rescanning make a fragmented large line quadratic in its size for a fixed chunk size. Paging by 50 turns does not bound the number of response bytes: a small number of turns can include very large tool results.

In an isolated copy of one affected session, native thread/resume finished in approximately 6 seconds, followed by one thread/turns/list response of approximately 68.7 MB (25 turns, 7,565 items). Reading that response through the existing adapter reader over a real child-process pipe took 117.1 seconds, nearly exhausting the host's 120-second setup deadline. With chunk-local scanning and a single join per completed line, the same history-read phase took 1.76 seconds; all turns and items were retained. These are history-transport measurements, not end-to-end session-load timings. The downstream reader in both measurements already had the UTF-8 decoding fix from #397, so the performance change does not rely on changing decoding behavior.

No private transcript is needed to reproduce the transport bottleneck.

Self-contained reproduction

From a checkout after npm ci, run the following on main, then on the proposed fix:

node --import tsx --input-type=module <<'JS'
import assert from 'node:assert/strict';
import { PassThrough } from 'node:stream';
import { performance } from 'node:perf_hooks';
import { createJSONRPCReader } from './src/StdUtils.ts';

const stream = new PassThrough();
const size = 64 * 1024 * 1024;
const bytes = Buffer.from(JSON.stringify({ id: 1, result: 'x'.repeat(size) }) + '\n');
let messages = 0;
const subscription = createJSONRPCReader(stream).listen(message => {
  assert.equal(message.result.length, size);
  messages++;
});
try {
  const start = performance.now();
  for (let offset = 0; offset < bytes.length; offset += 65536) {
    stream.write(bytes.subarray(offset, offset + 65536));
  }
  assert.equal(messages, 1);
  console.log({ bytes: bytes.length, elapsedMs: performance.now() - start });
} finally {
  subscription.dispose();
  stream.destroy();
}
JS

On a local macOS machine with Node v26.5.0, this 67,108,885-byte synthetic response took 4,269 ms with current main, versus 52 ms with the fix. Absolute timings vary with the machine and pipe chunk sizes.

Proposed fix

Scan only each newly decoded chunk for newline boundaries, retain unfinished fragments, and join once when a complete line is available. Clear retained fragments on disposal. This preserves complete history, framing, malformed-line handling, and the existing session setup deadline.

This is complementary to #397 (split UTF-8 decoding). It also affects large-session loading, but differs from #516: the bottleneck here occurs while reading an app-server response, before the history is published to an ACP client.