Skip to content

Batch local activity writes within row and byte limits - #1110

Merged
rmcdaniel merged 2 commits into
mainfrom
feat/general-1068-local-record-batches
Nov 9, 2026
Merged

rmcdaniel merged 2 commits into
mainfrom
feat/general-1068-local-record-batches

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Completions with adjacent local activity records currently issue an execution INSERT, an attempt INSERT, and separate sequence allocations and history INSERTs for each event. Buffer those rows within the existing transaction and flush each local run in bounded statements before another command, a history-dependent read or commit.

Multi-row INSERTs are limited to 100 rows and a conservative 1 MiB encoded-size target, including SQL/parameter overhead. A previously legal row above that target is written alone. Canonical history ranges use one allocation; the existing typed single-event fallback and logical completion-budget charges are preserved. Rollback drops pending rows.

Part of #1068. The issue stays open through qualification and publication. No benchmark speedup is claimed. The 3.3.1 release cut remains unchanged.

Validation covers 40-record order with an intervening side effect; physical INSERT/range counts on SQLite, MySQL/MariaDB and PostgreSQL; large-result splitting; oversized single-row preservation; event/row/byte refusals before local writes with the claim reusable; and range boundary/missing/noncanonical controls. Existing retry, heartbeat, failure and source parity tests remain.

Attempt 1 at 514dfcd compiled but was NONPASS: the direct registration fixture omitted its capability manifest, stopping the six backend native-test lanes before the new batch assertions. That result remains recorded. Successor f40ac3b adds only the supported local-activity manifest to that fixture; production and every assertion are unchanged. The successor passed all six original workflow runs with 62 successful checks, one intended skip and zero statuses. Source acceptance and queue selection. It entered the ordinary merge queue once at05:36:56UTC; integration/main and publication acceptance remain pending. Static whitespace, secret, ancestry and delta checks passed; no local application test or build ran.

Hold local execution, attempt and history rows in the transaction, then
flush bounded INSERTs before other commands and history-dependent reads.
Allocate canonical history ranges in one statement while keeping the
existing typed single-event fallback and logical persistence charges.

Add order, physical-write, large-result, oversized-row, budget refusal
and sequence-boundary controls across the existing database suites.

Co-Authored-By: Codex <noreply@openai.com>
The direct Store registration requires the canonical capability manifest
that HTTP registration normally supplies. Keep the production guard and
all batching assertions unchanged.

Co-Authored-By: Codex <noreply@openai.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

#1110 head514dfcda: interim NONPASS for the new test fixture; the original train is still running and nothing has been rerun or cancelled.

Production and lib-test targets compiled. The captured SQLite, PostgreSQL17 and MariaDB failures all stop at local_write_tests.rs:34 with409 worker_capability_not_supported. My direct Store registration supplied the capability list but omitted the canonical capability_manifest that HTTP registration supplies. The production guard correctly refuses it. The existing caught stale-history assertion in poll_transaction_counts is its deliberate #1093 control, not this failure.

Local successor f40ac3b adds only capability_manifest.local_activities.supported=true to that test registration (2additions/1deletion;1087B / SHA-25662be6892105184c1fe448b6b07196d63bcbe7f705006b9c32da7197110a73b6e). Production bytes and every assertion are unchanged. The successor is unpushed. All original jobs must finish; I will classify every terminal failure before selecting another normal train. The primary3.3.1 release and Sample App follow-through remain ahead of this held test-only successor.

@rmcdaniel

Copy link
Copy Markdown
Member Author

#1110 attempt1 at514dfcda is terminal NONPASS:55success,7failure,one intended skip. All six original workflows are terminal on attempt1; no cancellation or rerun.

All seven failed jobs are accounted for: the SQLite, both PG and all three MySQL/MariaDB native-test lanes each have exactly one failed test, stopping at local_write_tests.rs:34 with worker_capability_not_supported because my direct registration fixture omitted capability_manifest. The seventh failure is the aggregate. Captured logs confirm the same cause for every lane. The production and lib-test targets compile; no new batching-file compiler warning was found. Other source, corpus, restart, cohort, SDK-direction, security and boundary checks passed.

The new physical-count, large/oversized-row and budget assertions were not reached, so they remain unqualified. The local successor f40ac3b adds only the fixture's declared local-activity manifest; production and all assertions are unchanged. It remains unpushed, behind the active3.3.1 qualification and Sample App follow-through. No queue or publication for #1068 is selected. This original NONPASS remains attributed.

@rmcdaniel

Copy link
Copy Markdown
Member Author

General selects one successor push and fresh normal CI run on the existing draft #1110: f40ac3b5964f3f291d199c3de7533bf86c9997bf, tree caec590215f8e74509c2c144b4335a3fb364f198, sole parent 514dfcda, base/current main 603f9f5c.

The successor is test-only: 1,087 bytes / SHA256 62be6892105184c1fe448b6b07196d63bcbe7f705006b9c32da7197110a73b6e, adding the supported local-activity capability manifest to the direct registration fixture. Production and every assertion are unchanged. The complete delta against main is 49117 bytes / SHA256 4c024d9eafb4c93f7807cddca75b74ee7e7f4e08137bbf359e882406b92333d4.

Attempt 1 is terminal NONPASS and remains attributed. All original runs are complete, so the exact old-head leased push cancels nothing. Native 3.3.1 qualification and Sample App consumer follow-through have passed, clearing this successor's hold; the separately selected alias promotion changes no source. One push, no rerun or cancellation. This does not select ready/queue/merge, a tag or publication for #1068, and it does not change the 3.3.1 cut. The physical-count, large-row and budget controls still need execution on this head.

@sonarqubecloud

sonarqubecloud Bot commented Nov 9, 2026

Copy link
Copy Markdown

@rmcdaniel

Copy link
Copy Markdown
Member Author

Attempt 2 is terminal SOURCE PASS on f40ac3b5964f3f291d199c3de7533bf86c9997bf, tree caec590215f8e74509c2c144b4335a3fb364f198, base 603f9f5cbf4684fc3fba7dbf20b4ee115fe40453.

62 checks succeeded, one intended path-gated skip, zero statuses. All six original workflow runs succeeded on attempt 1; no rerun or cancellation. Shared fixtures, PHPUnit/contracts, Sonar, CodeQL and both Boundary runs passed.

All supported backend lanes passed. Retained artifact tails show SQLite execution 749/0/1; MySQL8.0/8.4 and MariaDB10.11 lib42/0 and execution A374/0/1; PostgreSQL16/17 lib34/0 and execution A374/0/1. The new long-run local-activity ordering test is named ok in the execution tails. The maintained lib runner invokes the commit/sequence qualification containing the new physical INSERT/range, large-row splitting, legal oversized-row and pre-write budget/reusable-claim assertions. The PG retained tails name that qualification ok; MySQL tails retain the all42-pass result, not every name. No missing tail is described as a separately observed result.

The successor changes only the test registration's canonical capability manifest; production and assertions are unchanged. Full delta: 49,117 bytes / SHA-256 4c024d9eafb4c93f7807cddca75b74ee7e7f4e08137bbf359e882406b92333d4. Attempt 1 remains NONPASS for the fixture omission, not infrastructure.

General selects one ordinary ready/queue entry at this exact head. Require unchanged base/current main, clean worktree, full delta, all current checks terminal/accepted, zero statuses and an empty queue. Watch only for up to45minutes; if still queued then, one exact dequeue and stop this entry. No queue bypass or re-entry. On merge verify signature, ordered parents, exact tree, original actual-main terminal checks and topic removal after ancestry. Release and published-artifact acceptance remain separate; Native3.3.1 excludes this change. #1075 stays local until this integration completes.

@rmcdaniel
rmcdaniel marked this pull request as ready for review November 9, 2026 05:36
@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 9, 2026
Merged via the queue into main with commit 0b383e5 Nov 9, 2026
63 checks passed
@rmcdaniel

Copy link
Copy Markdown
Member Author

#1110 merged once at06:00:53UTC as 0b383e55e9b2b324e747c49c1c8bc1d273f2d846, within the queue ceiling. No dequeue or re-entry.

Signature valid; ordered parents [603f9f5c,f40ac3b5]; tree caec590215f8e74509c2c144b4335a3fb364f198 equals the reviewed source. Main currently equals that merge. Original actual-main checks are running; terminal custody and topic cleanup are still pending. Keep the topic until that result. Source #1075 remains local. Native3.3.1/source0bcadca1 and its published index are unchanged; #1068 remains open through the later release and published-artifact acceptance.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Actual-main qualification has an infrastructure NONPASS; it is not terminally accepted yet.

CodeQL run42570529805, attempt1, main 0b383e55, failed only Analyze(rust), job129006612891. Initialize CodeQL received HTTP504 on its initial streaming download at06:01:21 and all fallback downloads, then stopped06:02:20. No Rust analysis ran; Python/JavaScript/Actions analyses passed. This is not a discovered product defect or a Composer install failure. Original job log23,479bytes / SHA-256 5e8ba7ed74d0a92b17ac968b9a74792067affdc3ac98b512848ec8ea4b9bfba8 is retained privately.

GitHub's official new timeout incident, c9px7cw2ghrs, opened05:59:07UTC and is investigating increased anonymous-user timeouts. Its incident endpoint is newer than the currently cached summary endpoint. The bundle failure is consistent with that incident; no broader causal claim is needed.

The other original main workflows are still running and will finish normally. No cancel/rerun has occurred. Keep the topic. #1075 has only a local, patch-identical rebase on this main and remains unpushed. Next: classify every original main result; after upstream recovery, select at most one exact failed-job retry on unchanged source, retaining attempt1 as NONPASS. Do not call main qualified from the passing PR/merge-group checks.

@rmcdaniel

Copy link
Copy Markdown
Member Author

General selects one exact failed-job retry of main CodeQL Rust, job129006612891 in run42570529805, unchanged main 0b383e55e9b2b324e747c49c1c8bc1d273f2d846.

All other original actual-main workflows are terminal success (the context-only Merge queue Sonar workflow is intentionally skipped). Attempt1 of this CodeQL run is terminal failure from the verified bundle-download504; its three other language jobs passed. No source/security finding is waived.

The vendor incident is still investigating. Its continued global status is not claimed recovered. A fresh06:37:48UTC probe of the same failed bundle URL now returnsHTTP206 and65,536actual bytes with the correct Zstandard magic, zero curl retries/errors. This proves dependency-path availability only, not full bundle integrity or Rust analysis. It justifies one finite attempt instead of blindly replaying the full source train or waiting for a global status page change.

Use the failed-job endpoint once, no source/policy/token change, no other workflow rerun. Preserve attempt1 as NONPASS. Verify attempt2 runs the same Rust job/head and requires normal initialization, analysis and upload success. Report byT0+20min; if the selected attempt is still live then, one exact cancel after re-binding run/attempt/head, with no further retry. #1075 remains local until current main's required analysis is obtained. Native3.3.1 remains unchanged.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The selected failed-job retry was rejected before any rerun. GitHub returned HTTP403, “Jobs in this workflow run cannot be re-run.” Readback still shows run42570529805 attempt1, completed/failure; no attempt2 exists and no retry cancellation is due. The original bundle-download NONPASS stays attributed.

All actual-main product workflows passed on unchanged main 0b383e55e9b2b324e747c49c1c8bc1d273f2d846; Rust CodeQL analysis is still missing. I checked GitHub’s documented default-setup administration endpoint and the current configuration. I will make one supported request retaining state=configured, with no language, query-suite, threat-model, runner or schedule change. This is an idempotent configuration application, not a claimed scan: only a returned run and its verified head/results can provide new evidence. A no-op or refusal ends this route. No disable/re-enable, advanced-workflow switch, empty source commit, or alert dismissal is selected.

The configuration before the request is: actions, JavaScript/TypeScript, Python and Rust; default query suite, remote threat model, standard runner, weekly schedule. #1075 remains local while this is resolved. Native3.3.1 is unchanged.

API: https://docs.github.com/en/rest/code-scanning/code-scanning#update-a-code-scanning-default-setup-configuration

@rmcdaniel

Copy link
Copy Markdown
Member Author

The supported same-state default-setup request was a no-op, not a fresh scan. HTTP202 returned setup run39635208843 from2026-10-21 at447bdbc5; it does not bind this main. Readback preserves every current analysis setting and updated_at; the run list contains no new scan of0b383e55. No further configuration change is selected.

Main product workflows are all terminal success. The original managed CodeQL run42570529805 remains NONPASS from its pre-analysis Rust bundle504; Python, JavaScript and Actions passed. The job retry was rejected, not executed. This missing Rust analysis remains explicit; neither old setup success nor PR/queue success substitutes for it.

I am revising my own sequencing decision to avoid parking authorized source work behind a managed workflow that cannot be retried: #1075 may begin its ordinary draft-PR train on0b383e55. That candidate must obtain the full normal checks, including Rust CodeQL, before integration. This changes no required check, scanning configuration, alert disposition, release gate or product source to provoke a scan. No empty commit or security disable/re-enable. #1068's source branch is retained until this outstanding main analysis is resolved; the delivered3.3.1 tag/image remain unchanged. #1068 itself is not released yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant