You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 9bdc6d3
Browse filesBrowse the repository at this point in the historyBrowse files
fix(spec): a stack whose mapping authors connectorSource validates and lints again — the live ledger row carries no author warning (#21127) (#21176)
Fixes#21127
Clause-②: no
## What this changes
`os validate --json` and `os lint --json` exited 1 on any stack whose
`mappings[]` entry authors `connectorSource`. The whole answer was the
liveness lint's integrity sentinel: `lintLivenessProperties: ledger
entry has unrecognised status "live"`. The ledger row
`mapping.connectorSource` had been re-graded `live` and kept
`authorWarn: true`. `describe()` in
`packages/lint/src/lint-liveness-properties.ts` throws on a warned
`live` row by design, and that throw stays as it is.
- **The row** (`packages/spec/liveness/mapping.json`): `connectorSource`
stays `live` and drops `authorWarn` / `authorHint`. Its `note` records
why.
- **The caveat moves to where an author reads it**
(`packages/spec/src/data/mapping.zod.ts`). The `connectorSource`
`.describe()` now reads: "Pulled when a job drives it; nothing schedules
it yet, so the binding alone moves no rows — schedule the pull with a
`job` once a job can drive one". The docblock no longer says the ledger
keeps `authorWarn`.
- **The existing integrity check refuses the combination, inside its
current gate** (`check:liveness`,
`packages/spec/scripts/liveness/check-liveness.mts`). No new gate or
script was added. The graded walk never reads a container row that
drills into `children`, and `connectorSource` is such a row, so the new
rule walks the raw ledger rows at every depth. It fails on `status:
"live"` with `authorWarn: true`, prints a prescription, and prints a
census line on every run: `author warnings: 2 ledger row(s) opt into
authorWarn, at any depth (planned 2); 0 on a live row.`
- **Regenerated:** `content/docs/references/data/mapping.mdx` and
`integration/connector.mdx` (`gen:docs`), and
`packages/spec/src/migrations/registry.ts` (`gen:migration-registry`).
## Files beyond the claim's listed surface (each one is text this change
made false)
The claim lists the ledger row, the `connectorSource` describe, the
existing integrity check, the regenerated artefacts, the pins and one
changeset. These files are outside that list. Each one said that
authoring the binding warns, and that stopped being true with the row
fix:
- `packages/spec/src/integration/connector.zod.ts`: the retired
`connector.syncConfig` prescription, which an author sees in the
refusal.
-
`packages/spec/src/migrations/entries/semantic/18.connector-sync-keys-retired.ts`:
the upgrade entry's acceptance text. `registry.ts` is regenerated from
it.
- `packages/spec/src/integration/connector-sync-retirement.test.ts`: a
one-line pin on that prescription text.
- `packages/spec/docs/SYNC_ARCHITECTURE.md`: "authoring
`connectorSource` still warns".
- `packages/spec/liveness/README.md`: the mapping row's note, plus a
third `authorWarn` rule (a `live` row carries none).
- `packages/lint/src/authoring-rules.ts`: comments only.
- `packages/lint/src/runtime-gate.inert-type-writes.test.ts`: a comment,
and a pin. A `mapping` write that authors `connectorSource` used to get
an `authoring-rule-threw` advisory at the runtime door, and now gets
none.
- `packages/cli/test/validate-lint-mapping-connector-source.test.ts`:
the door pin, a new file.
- `.changeset/20919-spec-connector-source-live.md`: see the next
section.
## ⚠️ One deliberate correction of a pending release note (needs
confirming)
`.changeset/20919-spec-connector-source-live.md` belongs to PR #21084.
It has not been released yet, and it said "`connectorSource` rows are
`live` and keep `authorWarn`". This PR makes that false, so the sentence
now reads "rows are `live`, with no author warning: that nothing
schedules a pull yet is said on the key's description".
`node scripts/check-empty-changeset.mjs --base origin/main` is therefore
red by design. It says: "This PR changes a changeset it did not add …
DELIBERATE CORRECTION -- … do NOT restore it -- say so on the PR and get
it confirmed". This section is that statement, and `skip-changeset` is
not applied.
`.changeset/20281-connector-sync-moved-to-mapping.md` carried a similar
claim. PR #21150 has since rewritten it on `main` without the warn
claim, and this PR does not touch it.
## Premise check
- **H1, the crash** (measured at the dispatch base `665cab338f`, which
contains PR #21092 `b616c0a63d`). The card's fixture is one object
`fx_account` with `sharingModel: 'private'`, plus one mapping with
`sourceFormat: 'json'`, `targetObject: 'fx_account'`, `mode: 'upsert'`,
a `fieldMapping` and `connectorSource: { connector: 'crm_api', action:
'request' }`.
- At base, `os validate --json` and `os lint --json` both exited 1, and
the only error was the sentinel.
- After the fix, on the merged head `b6d58dd04`: `validate` exits 0 with
`"valid": true`. `lint` exits 0 with `"passed": true` and one unrelated
warning (`protocol/missing-engines-range`). Neither output names
`connectorSource` or contains the sentinel.
- The crash had been hiding a real finding. Without `sharingModel`, the
same object gets `security-owd-unset`, so the fixture declares it.
- **H2, the integrity check.** Yes, inside `check:liveness`. Run against
the unfixed ledger, the gate exits 1 with exactly one ✗ block: "1 `live`
ledger row(s) opt into `authorWarn` — the author-side lint throws on
them: mapping/connectorSource".
- **H3, census** (re-run on the merged head). Across all 41 governed
ledgers, 924 rows were walked at every depth. Two carry `authorWarn:
true`, both `planned` (`object.externalSharingModel`,
`translation.flows`). None is `live`. Before the fix,
`mapping.connectorSource` was the only `live` one, so no other row has
this defect.
- **H4, describe text.** Done as above. Author-facing text cites no
tracker number.
## Pins
- `packages/spec/src/data/mapping-connector-source.test.ts`: every key
of the binding is `live`, and no row of it, at any depth, carries
`authorWarn` / `authorHint`. The description still says nothing
schedules a pull. The old assertion `authorWarn === true` pinned the
defect.
- `packages/spec/scripts/liveness/check-liveness.test.ts` runs the real
gate via `--ledger-root`:
- green on the shipped ledgers, with the census line showing warned
`planned` rows and none on a `live` row (the control);
- red when a `live` container row with `children` opts in (the
regression's shape);
- red when a drilled `live` child opts in.
- The carriers are picked from the shipped ledgers by shape, not by
name.
- `packages/lint/src/lint-liveness-properties.test.ts` runs against the
real ledger. The card's fixture lints without throwing and draws no
`connectorSource` finding. Control: `object.externalSharingModel`
(`planned` + `authorWarn`) still warns as `liveness-planned-property`.
- `packages/lint/src/runtime-gate.inert-type-writes.test.ts`: a
`mapping` write that authors `connectorSource` gets no errors and no
advisories at the runtime door.
- `packages/cli/test/validate-lint-mapping-connector-source.test.ts` is
in the integration tier and spawns the CLI:
- the card's fixture: `os validate --json` exits 0 with `valid: true`,
and `os lint --json` exits 0 with `passed: true`;
- control: adding `externalSharingModel` produces a
`liveness-planned-property` warning, still at exit 0.
## Reverse verification (one-time, not a permanent file)
This was run twice on the committed fix, with identical results: once at
`7bc7c5c26`, and again after the `main` merge at `b6d58dd04`. `node
scripts/ablation-replace.mjs` put `"authorWarn": true` back on the
`connectorSource` row. The anchor count went 1 → 0, and the blob changed
`49cc72e394b8` → `65665b90b21d`. With the defect back, every leg went
red:
- door: `validate` exit 1 and `lint` exit 1, with the sentinel in both;
- `check:liveness` exit 1 (`1 on a live row — FORBIDDEN`);
- spec pins: 2 failed;
- lint pins: 3 failed;
- CLI door pin: 3 failed.
The tool restored the file and proved it: the blob equals HEAD and `git
diff HEAD` is empty. The ledger is read from `packages/spec/liveness/`
at runtime, not from `dist/`, so no rebuild was involved.
## Verification (all at `b6d58dd04`, the PR head)
`origin/main` `fde553c50` was merged in with `bash
scripts/pm/os-regen-merge.sh`. The merge was clean. `registry.ts`
auto-merged as text, and `check:migration-registry` then confirmed it.
Main had not touched the two reference pages, so they kept the branch's
bytes. No regeneration commit was needed: after a fresh spec build,
`check:generated` reports all 15 generated artefacts up to date. The
delta against `main` is unchanged: 19 files, +538 / −42.
| what | command | result |
|---|---|---|
| spec build | `pnpm --filter @objectstack/spec build` | exit 0 |
| generated artefacts | `pnpm --filter @objectstack/spec run
check:generated` | exit 0, "All 15 generated artifacts are up to date" |
| CLI closure build | `pnpm exec turbo run build
--filter=@objectstack/cli... --concurrency=2` | exit 0, 59/59 tasks |
| the card's fixture | `os validate --json` / `os lint --json` | exit 0
/ exit 0, sentinel 0 times, `connectorSource` 0 times |
| ledger integrity | `pnpm --filter @objectstack/spec run
check:liveness` | exit 0, "0 on a `live` row" |
| strictness ledger | `pnpm --filter @objectstack/spec run
check:strictness-ledger` | exit 0 |
| spec tests | `vitest run scripts/liveness src/data
src/integration/connector-sync-retirement.test.ts src/migrations` | exit
0, 130 files, 4269 passed, 1 todo |
| spec typecheck | `pnpm --filter @objectstack/spec typecheck` | exit 0
|
| lint, whole package | `pnpm --filter @objectstack/lint exec vitest
run` + `typecheck` | exit 0, 118 files, 5489 passed; typecheck exit 0 |
| CLI unit tier | `vitest run --project unit test/lint test/validate
src/commands/validate src/commands/lint` | exit 0, 11 files, 125 passed
|
| CLI door pin | `vitest run --project integration
test/validate-lint-mapping-connector-source.test.ts` | exit 0, 3/3 |
| eslint, narrowed | `pnpm exec eslint --no-inline-config --format json`
on all 19 changed paths | exit 0. eslint linted 12 and reported 7 (`.md`
/ `.mdx` / `.json`) as "no matching configuration"; 0 errors, 0 warnings
|
| derived gates | `dispatch-gates.mjs --commands` (117), each run, then
`--ran` | 117 run: 116 exit 0, and 1 exit 1. That one is
`check-empty-changeset --base origin/main`, the deliberate correction
above. `--ran`: "117 run, 0 NOT-MEASURED, 0 UNRUN".
`check:pm-dispatch-gates` was killed by a timeout at 400s and then at
540s. It was then run to completion and exited 0; its battery took 1432s
on this box |
**The eslint narrowing is a measurement, not a skip:**
- The population comes from eslint's own answer for each changed path:
12 linted, and 7 that `eslint.config.mjs` has no configuration for.
- The file count comes from the `--format json` output.
- The config enables no type-aware linting (no `parserOptions.project`),
so this diff cannot move the verdict on any file it did not touch.
## Acceptance notes
- **turbo 2.11.5 wrote into `AGENTS.md` before the merge.** It appended
its managed `turborepo-agent-rules` block on every AI-attributed turbo
run in this worktree: 3 builds, and once during `check:type-check-debt`.
Each time the file was restored with `git checkout HEAD -- AGENTS.md`,
and `git diff HEAD` was confirmed empty. Files were staged by path only,
so the block was never staged and is not in this PR. PR #21151
(`agentGuidance: false` in `turbo.json`) came in with the merge. Since
then, `AGENTS.md` stayed unmodified across the 59-task CLI closure build
and all 117 gates.
- `@objectstack/lint` gets no changeset. Its source change is comments
only. Measured: tsup keeps comments, so the comment text does reach
`dist/*.js` bytes. The seat can override this.
- #21127 is cited only where agents and reviewers read: the
`authoring-rules.ts` comment, and the ledger's `note` and README prose.
Author-facing text (`.describe()`, the retired-key prescription, the
upgrade entry) cites no tracker.
---
_Generated by [Claude
Code](https://claude.ai/code/session_017VaLJnYwhPsanVCe9dMCJU)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
fix(spec): a stack whose mapping authors `connectorSource` validates and lints again — the liveness ledger's `live` row no longer carries an author warning
6
+
7
+
Clause-②: no
8
+
9
+
`os validate` and `os lint` exited 1 on any stack with a `mappings[]` entry that
10
+
authored `connectorSource`, and the only output was the liveness lint's internal
11
+
error `ledger entry has unrecognised status "live"`. The ledger graded the key
12
+
`live` (the connector sync executor reads every key of the binding) and still
13
+
asked the lint to warn whoever authored it; the lint has no warning for a key
14
+
that works, and stops on that inconsistency by design. The row carries no warning
15
+
now, so both commands judge the stack and exit 0 when nothing else is wrong. The
16
+
runtime metadata door no longer returns an `authoring-rule-threw` advisory for
17
+
the same mapping.
18
+
19
+
The note the warning used to carry is on the key's description, where an author
20
+
reads it: a pull runs when a `job` drives it, nothing schedules one yet, so the
21
+
binding alone moves no rows. The retired `connector.syncConfig` prescription and
22
+
the `connector-sync-keys-retired` upgrade entry no longer say that authoring the
23
+
binding warns.
24
+
25
+
`check:liveness` now refuses a `live` ledger row with `authorWarn: true` at any
26
+
depth, and prints how many rows opt into an author warning on every run. A
27
+
`planned` row with `authorWarn` still warns. No key, value or default changed.
|**upsertKey**|`string[]`| optional | Fields to match for upsert (e.g. email) |
52
-
|**connectorSource**|`{ connector: string; action: string; input?: Record<string, any>; recordsPath?: string; … }`| optional | Pull binding: the rest/openapi connector this mapping pulls rows from (one-way, full or timestamp-incremental; a `job` sets the cadence). Pulled when a job drives it; nothing schedules it yet |
52
+
|**connectorSource**|`{ connector: string; action: string; input?: Record<string, any>; recordsPath?: string; … }`| optional | Pull binding: the rest/openapi connector this mapping pulls rows from (one-way, full or timestamp-incremental; a `job` sets the cadence). Pulled when a job drives it; nothing schedules it yet, so the binding alone moves no rows — schedule the pull with a `job` once a job can drive one|
|**triggers**|`never`| optional |[REMOVED]`connector.triggers` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — a connector trigger never started anything: `AutomationEngine.registerConnector` registers a connector's actions only, no polling loop read `intervalSeconds` (or the `interval` spelling it was renamed from), and no receiver was driven by a `webhook` trigger. Delete the key; the `ConnectorTrigger` shape leaves with it. To start work from an external system, write a flow that calls the connector's action in a `connector_action` node: for an external event, an `api` flow that the event's sender calls; for a scheduled pull, a `schedule` flow. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
194
-
|**syncConfig**|`never`| optional |[REMOVED]`connector.syncConfig` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever ran a connector-attached sync: nothing read `strategy`, `direction`, `realtimeSync`, `timestampField`, `conflictResolution`, `batchSize`, `deleteMode` or `filters`, so the `latest_wins` and `soft_delete` defaults resolved and deleted nothing. Delete the key; the `DataSyncConfig` shape leaves with it. A sync is defined on its TARGET: a `mapping` (`targetObject`, `fieldMapping`, `mode`, `upsertKey`) whose `connectorSource` names the `rest` or `openapi` connector it pulls from, the read action and an optional timestamp `watermark`, with a `job` for the cadence. Its pull runs when a `job` drives it; nothing schedules one yet, and authoring the binding warns until a job can. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
194
+
|**syncConfig**|`never`| optional |[REMOVED]`connector.syncConfig` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever ran a connector-attached sync: nothing read `strategy`, `direction`, `realtimeSync`, `timestampField`, `conflictResolution`, `batchSize`, `deleteMode` or `filters`, so the `latest_wins` and `soft_delete` defaults resolved and deleted nothing. Delete the key; the `DataSyncConfig` shape leaves with it. A sync is defined on its TARGET: a `mapping` (`targetObject`, `fieldMapping`, `mode`, `upsertKey`) whose `connectorSource` names the `rest` or `openapi` connector it pulls from, the read action and an optional timestamp `watermark`, with a `job` for the cadence. Its pull runs when a `job` drives it; nothing schedules one yet, so the binding alone moves no rows. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
195
195
|**fieldMappings**|`never`| optional |[REMOVED]`connector.fieldMappings` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever moved a value through a connector field mapping: nothing read `source`, `target`, `defaultValue`, `dataType`, `required` or `syncMode`. Delete the key; the `ConnectorFieldMapping` shape leaves with it. Map fields on the sync's TARGET instead: a `mapping`'s `fieldMapping` (`source` → `target`, with a `transform` the import path executes), which its `connectorSource` pulls through. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
196
196
|**webhooks**|`never`| optional |[REMOVED]`connector.webhooks` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — a webhook nested inside a connector was never registered as a `webhook` item, so it was never materialized into `sys_webhook` and never delivered, and nothing emits the connector events its `events` list could name (`sync.completed`, `auth.expired` and the rest). Delete the key; the nested shape leaves with it (`WebhookConfig`, `WebhookEvent`, `WebhookSignatureAlgorithm`). To have a webhook actually sent, declare it in the stack's top-level `webhooks:` collection, which is materialized into `sys_webhook` and delivered on record events — note that doing so STARTS deliveries this connector never made. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
197
197
|**rateLimitConfig**|`never`| optional |[REMOVED]`connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared``RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to list the mechanical edits for existing sources; apply them by hand. |
|**triggers**|`never`| optional |[REMOVED]`connector.triggers` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — a connector trigger never started anything: `AutomationEngine.registerConnector` registers a connector's actions only, no polling loop read `intervalSeconds` (or the `interval` spelling it was renamed from), and no receiver was driven by a `webhook` trigger. Delete the key; the `ConnectorTrigger` shape leaves with it. To start work from an external system, write a flow that calls the connector's action in a `connector_action` node: for an external event, an `api` flow that the event's sender calls; for a scheduled pull, a `schedule` flow. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
491
-
|**syncConfig**|`never`| optional |[REMOVED]`connector.syncConfig` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever ran a connector-attached sync: nothing read `strategy`, `direction`, `realtimeSync`, `timestampField`, `conflictResolution`, `batchSize`, `deleteMode` or `filters`, so the `latest_wins` and `soft_delete` defaults resolved and deleted nothing. Delete the key; the `DataSyncConfig` shape leaves with it. A sync is defined on its TARGET: a `mapping` (`targetObject`, `fieldMapping`, `mode`, `upsertKey`) whose `connectorSource` names the `rest` or `openapi` connector it pulls from, the read action and an optional timestamp `watermark`, with a `job` for the cadence. Its pull runs when a `job` drives it; nothing schedules one yet, and authoring the binding warns until a job can. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
491
+
|**syncConfig**|`never`| optional |[REMOVED]`connector.syncConfig` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever ran a connector-attached sync: nothing read `strategy`, `direction`, `realtimeSync`, `timestampField`, `conflictResolution`, `batchSize`, `deleteMode` or `filters`, so the `latest_wins` and `soft_delete` defaults resolved and deleted nothing. Delete the key; the `DataSyncConfig` shape leaves with it. A sync is defined on its TARGET: a `mapping` (`targetObject`, `fieldMapping`, `mode`, `upsertKey`) whose `connectorSource` names the `rest` or `openapi` connector it pulls from, the read action and an optional timestamp `watermark`, with a `job` for the cadence. Its pull runs when a `job` drives it; nothing schedules one yet, so the binding alone moves no rows. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
492
492
|**fieldMappings**|`never`| optional |[REMOVED]`connector.fieldMappings` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever moved a value through a connector field mapping: nothing read `source`, `target`, `defaultValue`, `dataType`, `required` or `syncMode`. Delete the key; the `ConnectorFieldMapping` shape leaves with it. Map fields on the sync's TARGET instead: a `mapping`'s `fieldMapping` (`source` → `target`, with a `transform` the import path executes), which its `connectorSource` pulls through. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
493
493
|**webhooks**|`never`| optional |[REMOVED]`connector.webhooks` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — a webhook nested inside a connector was never registered as a `webhook` item, so it was never materialized into `sys_webhook` and never delivered, and nothing emits the connector events its `events` list could name (`sync.completed`, `auth.expired` and the rest). Delete the key; the nested shape leaves with it (`WebhookConfig`, `WebhookEvent`, `WebhookSignatureAlgorithm`). To have a webhook actually sent, declare it in the stack's top-level `webhooks:` collection, which is materialized into `sys_webhook` and delivered on record events — note that doing so STARTS deliveries this connector never made. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
494
494
|**rateLimitConfig**|`never`| optional |[REMOVED]`connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared``RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to list the mechanical edits for existing sources; apply them by hand. |
0 commit comments