Skip to content

Identify Delivery fallback 2 !== 1 assertion in Deno and Bun CI #1176

Open

Description

@taras

The unmodified Delivery wrapper fails during the runner-owned Bun weight measurement at commit 35749f4267d42449f26b6a18c4911a0ab7222923.

  • First failed Bun job, run 98 attempt 1: packages/cli/tests/adversarial-delivery.test.ts, “A-H required review follows Fable, one independent Astra replacement, then a retained Elicit hold”, 2 !== 1 after 80,209 ms.
  • Bun 1.4.0, Deno 2.9.5, Node v22.23.4 on ubuntu-latest (Ubuntu 26.04.1, image 20261101.186.1, agent 2.338.2).
  • Measurement stopped after 20 of 453 applicable Bun files. Deno completed all 563 files and Node all 454; assembly was skipped. Partial records are not imported or combined with another run.

The first full Bun log was preserved before any further execution: 411,436 bytes, SHA-256 b31df5909e007660c3f653b5bf9fc45e2abc79be62e20788a35d8fe4f90bc354. Original failed record artifact, ZIP SHA-256 8bcdf38255d1b689a2c36ac54b6a13cd4eff1f9aed03f3e90533ff331e82d1fe.

The original output does not identify which inner scalar assertion failed. Several assertions compare a count to 1; neither an overlap race nor any other cause is established.

A single instrumented full Linux capture passed at diagnostic envelope 63eef68dc359792133fce73a236c11db6c00171f, using a fresh exact product35749 checkout and four fixture instrumentation files. Admission matched the original image/runtime facts. All 16 cases completed, with 346 before/passed assertion pairs and no capture error; the full consumer exited0 in140.267s. First capture artifact, ZIP SHA-256 13875cf2a25f63884827459f8935fb08c427dff01b1ec2f37f69aadd020eacf7.

Instrumentation changes timing and the preceding measurement-file load was not recreated. This pass does not establish unchanged-revision intermittence or a causal fix. Earlier diagnostic setup/context/collector failures are separate from the original product failure and remain preserved.

Identify the failing assertion and its triggering execution boundary before changing production behavior. Preserve every fallback decision, independent-review replacement, retained Elicit hold, reservation, original assertion, all16cases, full Delivery context, deadlines and joined cleanup. Do not raise deadlines, exclude cases or accept another green run as a cause fix. Keep this issue open until cause and discriminating regression evidence are recorded. Related: PR#1174 / main-health#1158; Preparation#1117 is a separate timeout investigation.

Repeated uninstrumented failure on the identical revision

Fresh whole measurement run99, Bun job128614185373, attempt1, fails the same A-H group with literal 2 !== 1 at unchanged 35749f4267d42449f26b6a18c4911a0ab7222923. The inner scenario/assertion is still unnamed. The job checkout SHA and artifact provenance confirm the original fixture/consumer bytes; no diagnostic overlay ran. Bun again stopped after20/453 applicable files; Deno and Node continue independently. No further measurement is dispatched.

The complete first recurrence log is411,219bytes, SHA-256 81304dac2c11c43681dd00c3a9c9f443313d7acaebaaf865bcfdf3d66656017a. Original recurrence artifact is3,956bytes, ZIP SHA-256 3f3c33cbda8a68b276c9dd601a7d481859148dacf7160b529501ace81debe966. Both first raw channels and the exact failed JSON record are preserved. This is repeated failure, not same-revision failure/pass intermittence; the qualified instrumented pass above does not justify a flake label or causal claim.

Failure-only capture after the recorded prefix

A single additional diagnostic capture used signed envelope b91382fc7616d235003e2ad8c218f299287ccf55. Exact historical Linux admission and focused Deno/Bun preflights passed. A separate exact35749 checkout ran the 20 preceding Bun files in the recorded order, all passing in282.907s, before applying exactly two overlays: the command-scope failure plugin and its inner CLI argument. All2,298 tracked baseline hashes and171 example files were verified; original assertions, operands, cases and deadlines were preserved.

The full target passed once in171.448s. The failure-only capture retained10 metadata files containing29 handled failure observations; all29 source hashes matched the product. None identifies the original failing assertion or contains its scalar2/1 detail. There were no capture faults. Original capture artifact:264,924bytes, ZIP SHA-256 c0ddf94997315a7f67e001d4cd5939f8b2630e72d6db0577441274ada57bfa0c;143 nonself inventory hashes verified.

Replaying programs/order does not reproduce machine/cache/timing state, and instrumentation remains present. This is another qualified diagnostic pass, not an unmodified fail/pass pair, causal fix, or reason to close this issue or apply a flake label. No automatic retry or further whole measurement is authorized. Run99 Node has since completed454/454; Deno remains active. Its failed Bun record still prevents assembly.

Static-label capture and delivery separation

The single static-label capture, signed diagnostic envelope 30e2c40cfce24b4b272d7acf9708d25930ae68f6, passed exact Linux admission and focused controls. All20 original prefix files passed in276.026s before the sole Fallback.md overlay. Removing its65 static msg attributes exactly restores the original product blob; all2,298 baseline hashes/modes and the original Entry were checked. The target ran once with no diagnostic plugin or XMD_TEST_ARTIFACTS and passed in161.793s. No assertion label failed. Original artifact13606503164:282,278bytes, ZIP SHA-256 2651d3f12b9c0b292feb526c263e7aac17f30fcd6a6b303aa5bd6cb13b8b5d7e;139 nonself inventory hashes verified.

This remains a qualified diagnostic pass, with different labelled source, process/timing and owned temporary-directory context. It identifies no historical assertion or cause and supplies no unmodified fail/pass pair. The diagnostic episode stops without another probe or measurement. This issue stays open.

The actual current Main CI failures are the separate DT9 progress-location assertion and J4 cancellation-readiness test; their already-reviewed two-file repairs are being isolated for Main. This Bun assertion blocks coherent weights for the broader lifecycle/API delivery PR#1174; it is not one of those observed Main CI failures. Deno's independent whole-measurement budget variance is #1179. No dependency on those broader investigations is imposed on the unchanged-corpus Main test repair.

Deno CI recurrence after the shared CI repair

PR #1182 Deno shard 10, attempt 1, fails the same A-H group with literal 2 !== 1 at signed branch head aba33963d54d693687dc7be6ecaf575b90bfc986. The Delivery carrier took about 56 seconds; the shard continued through all 25 assigned files and exited 1 for packages/cli/tests/adversarial-delivery.test.ts. The exact inner scenario/assertion remains unknown. This recurrence is not confined to Bun.

The complete first raw job log and numeric downloader/hash receipt were preserved before any rerun. No rerun or new measurement was dispatched. The failing job explicitly reports no files under .xmd/test-artifacts; its test-journals-deno-10-attempt-1 artifact query is empty.

The current runAdversarialJourney routes only ProductionRole and Preparation through xmd test. Delivery still executes xmd run --eval <Testing>, and Fixture.Entry starts a separate xmd run for FallbackCase. Those boundaries bypass the CLI test command's failure-only diagnostics. This explains the missing automatic artifact in this run; it does not identify the scalar's cause.

Current-main documentation at 23c9003c84d9964ac85fb16ead1d2cb22350596c governs the bounded follow-up. Preserve all 16 cases, assertions, deadlines, fallback/hold semantics and joined cleanup. Obtain the exact leaf and discriminating evidence before changing behavior. Historical instrumented passes remain qualified; this issue stays open.

Unchanged Deno revision passed on one retry

The first failure above and this retry both ran commit aba33963d54d693687dc7be6ecaf575b90bfc986, Deno 2.9.5, shard 10/22. No source fix or diagnostic overlay was applied between them.

  • Attempt 1, failed job: Delivery A-H fallback batch failed with Expected values to be strictly deep-equal: 2 !== 1. No failure journal artifact was uploaded for this legacy xmd run path.
  • Attempt 2, passing job: deno shard 10/22: 25 of 25 files ran, all passed; Test step completed successfully at 2026-11-08 18:23:38 UTC. This was one failed-job retry on the same workflow/head, not a source change.
  • First failure log: 555827 bytes, SHA256 9998894f9be1921200ab14376594a4b2592d8dabf097ab54345c289cf9d1cae9.
  • Passing log retained locally: 521625 bytes, SHA256 51436c747324e7829e3f0bb0b7546b2e14cb95bb8f6d566ba8b1312a04d0adb7.

This establishes intermittence on an unchanged revision and warrants the flake label. It does not identify the original scenario/assertion or establish a causal repair. Keep this issue open until the cause and discriminating regression evidence are recorded. The proposed fixture-only failure-attribution migration remains held; no implementation was started.

Node CI recurrence in PR #1203

At unchanged PR head 3496f489312d500e1aa2543802ec07c8dd846083, CI run 42551180878, Node shard 11/18, attempt 1 fails the same A-H fallback group with Expected values to be strictly deep-equal: 2 !== 1. Command: pnpm test:node 11/18; Linux, Node 22.23.4, Deno 2.9.5. The complete shard still runs 44/44 files, then reports packages/cli/tests/adversarial-delivery.test.ts as its first failure. This recurrence is not confined to Deno and Bun.

The complete original job log was saved before any retry: 5,482,466 bytes, SHA-256 4308d8e14a3b9224711e79b1af1bc8d6fbbe3b6dde81ba6caa71a241a3c98331. Retained at /private/tmp/xmd-staged-submission-evidence-20261108/ci/node11-first.log. The uploaded test-journals-node-11-attempt-1 artifact contains only an unrelated expected-negative journal; it supplies no inner FallbackCase assertion attribution. No fallback production or fixture source changed in #1203.

The retry on that exact head passed as job 128951602366: node shard 11/18: 44 of 44 files ran, all passed, completed 2026-11-09 03:00:42 UTC. No source correction or diagnostic overlay was applied. The passing complete job log is 5,454,830 bytes, SHA-256 ba1d5bac479c105d3d07c71396c3ad10502df18b48c6312cd45b26c35430c84d, retained beside the original as node11-retry.log. This is an unchanged-revision fail/pass pair under Node; it establishes intermittence, not the exact leaf or a cause fix. A separate scratch-only local probe of five cases with failure labels passes, but it changes instrumentation/timing and is not a causal repair or a substitute for unchanged CI evidence. No further local fallback probe or broad measurement is planned.

PR #1203's separate ProductionRole outer-deadline failure is corrected by moving its three newly added submission cases, byte-for-byte, into their own bounded suite. That correction changes no fallback behavior or deadline. Keep this issue open for its existing exact-leaf/cause investigation; no original assertion or fallback/hold guarantee is weakened.

Activity

  1. changed the title [-]Identify Delivery fallback 2 !== 1 assertion in Bun weight measurement[/-] [+]Identify Delivery fallback 2 !== 1 assertion in Deno and Bun CI[/+] on Nov 8, 2026
  2. added
    flakeIntermittent or timing-sensitive failure that can pass without a code change
    on Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    flakeIntermittent or timing-sensitive failure that can pass without a code change

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions