fix(core,driver-memory,service-analytics): sum / avg add with one compensated sum on every face (#20544) - #20739
Conversation
…um / avg with the one compensated sum `compensatedSum` moves from objectql's rows path to `@objectstack/core` as a root export, as `bucketDateKey` did, and the three folds that still added naively call it: driver-memory's native `sum` / `avg` arm, its analytics face (a `$group` `$accumulator` in place of mingo's `$sum` / `$avg`), and service-analytics' draft preview. objectql imports the helper instead of keeping its own copy. Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
… avg on every face The card's `0.1 + 0.2 + 0.3` fixture, the `1e16` cancellation, a two-addend control and integers: on the core helper itself, on driver-memory's data face (both doors) and analytics face (plain, time-bucketed, ordered, and its pipeline dump), and on the draft preview as a differential against the live face on a real SQLite. Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…ers with valid identifiers Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…amily Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift Check10 anchor(s) derived from 4 changed package(s); no hand-written page names any of them. What this run could not see
Coarse fallback — 41 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin af50d27a14e153e88f8af7d9c16ee69550a2c5a1 && git checkout af50d27a14e153e88f8af7d9c16ee69550a2c5a1
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin fbec216e2d184afc03b456b0bd8013ad5d47bc6f b7e98273898d2247d48b36ee976137573f7a20b4 && git checkout -B drift-repro fbec216e2d184afc03b456b0bd8013ad5d47bc6f && git merge --no-ff b7e98273898d2247d48b36ee976137573f7a20b4
node scripts/docs-audit/affected-docs.mjs --json fbec216e2d184afc03b456b0bd8013ad5d47bc6f |
Contract reviewServed-tier: Inputs: card #20544 (body; comments 5882237833 triage, 5900737084 claim, 5901507878 os-dev-report), PR #20739 (body, its 10-file list, the net diff Check-runs on the head as read: 31 runs — 12 ① Derived judgmentsThe hoist is byte-identical — right. The nine-line body of The new export as a contract — right. Name Fold 1, driver-memory data face ( Fold 2, driver-memory analytics face ( The pipeline dump's function arm — right. Fold 3, service-analytics draft preview — right. PostgreSQL and MySQL untouched — right. No file under Pins per face — right. Each face carries the Shipped prose, sentence by sentence — true. Changeset: SQLite 3.43+ uses Kahan-Babuska-Neumaier for ② Semver level
③ Boundary flags
Residuals stated, none blocking: Implemented-by: VERDICT: PASS Generated by Claude Code |
⛔ merge queue 构建失败 — 先分诊,再决定要不要重排队列构建 36651106552 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集), 失败的 job(日志抽取,best effort):
跨 PR 相同签名(24h,按失败测试文件聚合):
历史信号:
分诊清单:
Generated by Claude Code · merge-queue-triage workflow (#4859) |
Fixes #20544
Clause-②: yes
What changed
sum/avgnow give the same double on every face the platform owns. One compensated fold does the adding everywhere.compensatedSummoves from@objectstack/objectql's rows path (in-memory-aggregation.ts, private,:317at based2820876f) topackages/core/src/utils/compensated-sum.ts. It is exported from the@objectstack/coreroot, besidebucketDateKey, which is thebucketDateKeyprecedent.in-memory-aggregation.tsimports it and no longer keeps a copy. The function body is byte-identical: it is SQLite'skahanBabuskaNeumaierStepplus the finalizers' overflow guard.packages/drivers/driver-memory/src/memory-driver.ts,computeAggregate'ssum/avgarm. This is the doorengine.aggregate's native path takes on driver-memory, andfind()with aggregations uses it too.packages/drivers/driver-memory/src/memory-analytics.ts,buildAggregator. Asum/avgmeasure is now one$group$accumulator, whosefinalizecallscompensatedSum. It replaces mingo's$sum/$avg. The aggregand expression (numericAggregandExpr, the boolean rule) is unchanged. So are the addend predicate (mingo'sisNumber) and the empty answers (sum0,avgnull).packages/services/service-analytics/src/preview-evaluator.ts, thesumarm and theavgarm. The operand lists are unchanged. Thedefault:arm and the region PR docs(service-analytics): re-anchor the dead tracker citations to the commits that decided them #20729 re-anchored are not touched.pipelineDumpReplacerrenders a function by its name. Without that,JSON.stringifydrops the accumulator's functions, and asummeasure and anavgmeasure would dump identically inresult.sql.sum/avgover 3+ fractional addends: SQLite native adds with compensation (0.1+0.2+0.3 = 0.6), every other face and the rows path naively (0.6000000000000001), sohaving $eq 0.6keeps the group on SQLite native only #20489's ruling.count,minandmaxare untouched.Why
Clause-②: yes: the hoist adds one root export (compensatedSum) to@objectstack/core, which widens its published surface. The changeset.changeset/20544-compensated-sum-every-face.mdis@objectstack/coreminor, withpatchfor@objectstack/objectql,@objectstack/driver-memoryand@objectstack/service-analytics.Reproduction: before and after
Scratch script (not committed), run with
tsxover the packages' sources and a rebuilt@objectstack/core. Setup: a driver-memoryObjectQLengine, onenumbercolumnw, one group per fixture.engine.aggregatewithsum/avg. A spy counted 1driver.aggregatecall.count. The spy counted 0driver.aggregatecalls.MemoryAnalyticsService.queryover a cube on the same table.evaluateAnalyticsQueryOverRowsover the same rows.Before, at base
d2820876f(sum/avg):0.1, 0.2, 0.31e16, 1, -1e160.1, 0.2(control)1, 2, 3, 40, 500(control)0.6/0.199999999999999981/0.33333333333333330.30000000000000004/0.15000000000000002546/109.2engine.aggregate)0.6000000000000001/0.200000000000000040/00.6000000000000001/0.200000000000000040/00.6000000000000001/0.200000000000000040/0having { s: { $eq: 0.6 } }throughengine.aggregatekept no group on the native path and keptcardon the rows path.After, at
37a825e0cwith core rebuilt: all four faces answer the rows-path row in every column.having { s: { $eq: 0.6 } }keepscardon both paths.The analytics face's measured
orderover thesummeasure changed fromcancel=0, two, card=0.6000000000000001, intstotwo, card=0.6, cancel=1, ints.Mechanism hypotheses: which held
compensatedSumwas private atin-memory-aggregation.ts:317, andbucketDateKeyreaches the core root throughexport * from './utils/datetime.js'(index.ts:50). Core'sexportsmap has only.and./logger. So the helper cannot be shared without widening the published surface: a new subpath would widen it too, and a copy per package is what triage ruled out. The lineClause-②: yes/ coreminorstands.Contextcannot replace$sum.Context.frommerges the built-ins first andaddOpskeeps an operator that is already there. A context whose own$sumreturns42still answered0.6000000000000001. A new operator name works ($mySumanswered42), but only in anAggregatorbuilt with that context. This face builds two: the driver's publicaggregate()and its own time-bucket half.$accumulatoris in the default operator set.ComputeOptions.initdefaultsscriptEnabledtotrue: the probe ran with default options, and withscriptEnabled: falseit refused ($accumulator requires 'scriptEnabled' option to be true).numericAggregandExpr's header already records: it runs after$sort/$limit.$accumulator. The time-bucketed pipeline andorderby the measure are pinned.memory-driver.ts:2022andpreview-evaluator.ts:514/:550at base.avgis the compensated sum divided by the count:0.19999999999999998on all four faces, which is the rows path's answer on the same values.count/min/maxarms are untouched, and integers are unchanged on every face (pinned). Stated boundary: this holds while the running total stays within 2^53. Beyond it the compensated total is the exact one, as PR fix(objectql): the rows path adds sum / avg with compensated summation, as SQLite does #20543 recorded for the rows path (2^53, 1, 1→9007199254740994). That boundary now applies to driver-memory's faces and the preview too.Tests
New pins. Each face gets the card's
0.1 + 0.2 + 0.3fixture, the1e16cancellation, a two-addend control and an integers control. Each asserts the naive fold's answer beside the expected one, so a fixture that cannot tell the folds apart fails.packages/core/src/utils/compensated-sum.test.ts: 6 cases, including the empty list and non-finite totals (Object.isagainst the naive answer).packages/drivers/driver-memory/src/memory-compensated-sum.test.ts: 12 cases.aggregate(AST),find(), the having reading, the addend rule, the empty group.orderby the measure, the addend rule, the empty group, and the dump naming each measure's fold.packages/services/service-analytics/src/__tests__/preview-compensated-sum.test.ts: 3 cases. It is a differential between the preview and the live face (NativeSQLStrategy's SQL on sql.js SQLite): twoAnalyticsServiceinstances that differ only indraftRowsResolver.in-memory-aggregation-compensated-sum.test.tsis unchanged and now runs through the core export.Suites and typecheck, at head
b7e98273(after mergingorigin/mainf927864ea):pnpm --filtertypecheck over@objectstack/core,@objectstack/objectql,@objectstack/driver-memoryand@objectstack/service-analytics: all fourDone. corecheck:test-typecheckOK (4 files / 4 errors held). objectqlcheck:test-typecheckOK (40 / 234 / 65 held).tsc --listFilescounts the three new test files once each in their packages' programs.--project local): 337 / 6687.e07690e1d, the counts were the same except objectql at 336 / 6679. Main added one objectql test file.test:repoprojects. Their files do not read this surface.Reverse verification (one-time, from committed state
e07690e1d)scripts/ablation-replace.mjsin wrap mode onpackages/core/src/utils/compensated-sum.ts. The anchorreturn Number.isFinite(c) ? s + c : s;becameconst ablation20544 = s; return ablation20544;, which is the naive running sum.30e811c70045→c2886a45b2d8.pnpm --filter @objectstack/core build, thenablation-dist-preflight.mjs @objectstack/core ablation20544found the marker in 2 built files. objectql's and service-analytics' suites resolve core throughdist/; driver-memory's aliases core tosrc/.git diff HEADempty.--absentfound the marker absent from all 14 built files, and the tree was clean.Gates
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack(no paths) atb7e98273derived 67 commands. All 67 were run.check:dual-build-cjs-loadsandcheck:type-check-debtfirst answeredPREREQUISITE NOT MET(exit 3, nodist/for other packages). Afterturbo run build --filter='./packages/*' --filter='./packages/*/*'(71 / 71 tasks), both exited 0.--ranwith exit codes:67 derived, 67 run, 0 NOT-MEASURED, 0 UNRUN(a derived zero).check:driver-conformance, before (based2820876f) and after (b7e98273):50 covered cell(s), 0 in the DEBT ledger, 0 exemptboth times. The dialect axis is unchanged.pnpm --filter @objectstack/spec check:api-surface:public API surface + factory signatures unchanged.@objectstack/speconly, and spec is untouched.@objectstack/core's exports. Its one-export widening is declared byClause-②: yesand theminorchangeset.check:adr-0087-registration(1 non-breaking changeset),check-changeset-no-major,check:empty-changeset,check:issue-citations(10 citations, all resolve),check:doc-authoring,check:nul-bytes,check:engine-double-contract,check:cross-package-test-inputs,check:test-source-aliasandcheck:undeclared-dep-imports.Lint: a declared narrowing, not a full run.
pnpm exec eslint --no-inline-config --format jsonover the 9 touched source files, atb7e98273, gave 9 files linted, 0 errors and 0 warnings.eslint.config.mjs:files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']and thepackages/**blocks, minusNEVER_LINTED. All 9.tsfiles are in it. The changeset.mdmatches nofilesglob.parserOptions.projectand no typed@typescript-eslintrules, as its own comment nearQUERY_OPTIONS_TEST_GLOBSstates. So this diff cannot move the verdict on any untouched file.pnpm lintis CI's.Acceptance notes
preview-evaluator.ts'sdefault:arm (custom-SQL metric types) still adds withreduce. The file records it as the historical answer with no live standard to move towards, and no card names it. Not changed. Carrier: none.driver-sql.ts'sAGGREGATE_ACCUMULATIONresidual note still points at the rows path (in-memory-aggregation.ts,compensatedSum). That is still true, because the rows path calls it by that name. The note does not mention that the helper now lives in core, or that driver-memory's faces use it too. It is outside this card's file surface, so it was not edited. Carrier: none.InMemoryDriver.aggregate(object, pipeline)with its own$sum/$avgstill gets mingo's plain loop. The platform's own producer of that arm (the analytics face) no longer emits them forsum/avgmeasures, andcountkeeps$sum: 1, whose integers are exact.preview-evaluator.ts(about:625) is untouched. That PR has landed onmain, and this branch merged it.Generated by Claude Code