From 680f4bb29b795544798b025f7931168da33daccf Mon Sep 17 00:00:00 2001 From: Rami Date: Thu, 22 Oct 2026 11:06:04 +0100 Subject: [PATCH 1/4] feat(definitions): a definition's overview is markdown its author gives beside the document ## What this changes `create_definition` and `update_definition` take an optional `overview` beside `source`: Markdown for the person who runs the business, at least one character and at most 16,384 bytes in UTF-8, declared once as `OverviewField` and bounded the way `SourceField` is. The version keeps it as given in its content, every command and `get_definition` answer it as written, and `list_definitions` leaves it out. An update that leaves it out keeps the saved overview: the registry takes it from the saved version where it decides the update, so the same document with the same or no overview records nothing, and a new overview alone records a new version. The presenter of a definition's facts measures it as written. The argument's description is the sentence of the ask that fits the 300 characters an argument's description may hold on MCP; the two commands' descriptions add that it is written again with every update and that one left out stays. The instructions of a connection that may create a definition ask for it in the sentence that points to the formats, within the 2,000 characters a connection's instructions may hold. ## What goes `src/overview` and `src/overview-words` of `@on.auto/definitions`, `src/used-by/overview-at-read.ts`, the `overview` hook of a capability with `OverviewOf`, `OverviewContext`, `composedOverview` and `PreparedDefinition.overview`, each capability's `src/overview` folder and its golden files, the `about` of every function's front matter and a workflow's `document.metadata.about`, and `DefinitionSummary.about`. A description is one or two sentences for lists, so the rules of `src/document/written-texts.ts` go, with `src/markdown`, which only they read, and the positions the front matter kept for them. The Liquid reading of `@on.auto/definitions/template`, the shape words, a workflow's diagram and step words, the title its summary kept and the text and line a reasoning prompt kept go with the modules that read them, and so do the exports only those modules used. The words of a schema that a refused value is told in move to `src/schema-words`, as plain text. A workflow numbers its steps with the engine's one walk of its tasks, depth first and the body before the recovery, the same numbers as before. The users of a definition stay in `src/used-by`, with their own `NamedDefinition` shape. The loads measure saves every version with an overview at its bound. ## Tests The registry keeps the saved overview through an update that leaves it out, keeps none where none was saved, records a new version for a new overview alone and nothing for the same document and overview. The operations answer the overview as written, refuse an empty one and one past 16,384 bytes in UTF-8 whatever its length in characters, and keep it across an update that leaves it out. The schema words hold the cases they held, as plain text. Co-Authored-By: Claude Fable 5.1 --- .oxfmtrc.json | 3 +- .../capability/computation-function.test.ts | 2 +- .../src/capability/computation-function.ts | 5 +- .../src/document/computation-document.ts | 1 - .../src/document/document-parsing.test.ts | 28 +- .../src/document/document-parsing.ts | 3 +- .../computation/src/document/front-matter.ts | 8 +- .../src/overview/computation-overview.test.ts | 145 --------- .../src/overview/computation-overview.ts | 14 - .../src/overview/month-window.overview.md | 37 --- .../overview/reference-example.overview.md | 25 -- .../src/overview/shaped-example.overview.md | 21 -- .../coordination/src/capability/workflow.ts | 2 - .../src/document/workflow-document.test.ts | 76 +---- .../src/document/workflow-document.ts | 30 +- .../src/document/workflow-functions.test.ts | 2 +- .../src/document/workflow-functions.ts | 2 +- .../src/document/workflow-summary.ts | 40 +-- .../src/document/workflow-texts.ts | 69 ---- .../coordination/src/overview/cron-words.ts | 66 ---- .../overview/prediction-example.overview.md | 47 --- .../overview/reference-example.overview.md | 59 ---- .../src/overview/shaped-example.overview.md | 41 --- .../src/overview/trigger-words.test.ts | 124 -------- .../src/overview/trigger-words.ts | 59 ---- .../src/overview/workflow-overview.test.ts | 161 ---------- .../src/overview/workflow-overview.ts | 46 --- .../coordination/src/steps/flow-words.ts | 105 ------ .../src/steps/grouped-tasks.test.ts | 155 --------- .../src/steps/incomplete-tasks.test.ts | 116 ------- .../coordination/src/steps/leaf-tasks.test.ts | 178 ----------- .../coordination/src/steps/placed-tasks.ts | 123 ------- .../coordination/src/steps/task-diagram.ts | 97 ------ .../coordination/src/steps/task-edges.ts | 174 ---------- .../coordination/src/steps/task-flow.test.ts | 143 --------- .../coordination/src/steps/task-flow.ts | 38 --- .../coordination/src/steps/task-labels.ts | 57 ---- .../coordination/src/steps/task-reading.ts | 57 ---- .../coordination/src/steps/task-words.ts | 79 ----- .../src/steps/tried-tasks.test.ts | 176 ----------- .../coordination/src/testing/done-steps.ts | 41 --- .../src/capability/definition-reading.ts | 11 +- .../src/capability/interaction-function.ts | 3 +- .../interaction/src/document/call-parts.ts | 1 - .../src/document/document-parsing.test.ts | 27 +- .../interaction/src/document/front-matter.ts | 8 +- .../src/document/interaction-document.ts | 1 - .../interaction/src/document/request-parts.ts | 1 - .../src/overview/approve-report.overview.md | 62 ---- .../interaction/src/overview/call-overview.ts | 13 - .../src/overview/interaction-overview.test.ts | 158 --------- .../src/overview/reference-call.overview.md | 23 -- .../src/overview/reference-chat.overview.md | 53 ---- .../src/overview/reference-inbox.overview.md | 35 -- .../src/overview/request-overview.test.ts | 163 ---------- .../src/overview/request-overview.ts | 125 -------- .../interaction/src/overview/request-party.ts | 16 - capabilities/interaction/src/testing/index.ts | 1 - .../prediction/measure/order-history.ts | 3 +- .../src/capability/prediction-definitions.ts | 3 +- .../src/capability/prediction-function.ts | 2 - .../src/dataset-testing/projected-datasets.ts | 7 +- .../src/document/document-parsing.test.ts | 23 +- .../src/document/document-parsing.ts | 1 - .../prediction/src/document/front-matter.ts | 8 +- .../src/document/prediction-document.ts | 1 - .../prediction/src/overview/data-words.ts | 92 ------ .../src/overview/evaluation-words.ts | 69 ---- .../src/overview/prediction-overview.test.ts | 172 ---------- .../src/overview/prediction-overview.ts | 33 -- .../prediction/src/overview/predicts-words.ts | 84 ----- .../overview/reference-example.overview.md | 62 ---- .../overview/reference-uploaded.overview.md | 55 ---- .../prediction/src/overview/trainer-words.ts | 31 -- .../src/capability/reasoning-function.test.ts | 2 +- .../src/capability/reasoning-function.ts | 2 - .../src/definition/definition-parsing.test.ts | 2 +- .../src/definition/definition-parsing.ts | 4 +- .../src/definition/definition-reading.ts | 2 - .../definition/front-matter-schema.test.ts | 30 +- .../src/definition/front-matter-schema.ts | 3 - .../reasoning-function-definition.ts | 1 - .../src/overview/reasoning-overview.test.ts | 209 ------------ .../src/overview/reasoning-overview.ts | 120 ------- .../overview/reference-example.overview.md | 31 -- .../src/overview/reference-tools.overview.md | 50 --- .../src/overview/shaped-example.overview.md | 31 -- .../src/template/compiled-template.ts | 3 +- .../src/template/template-compilation.test.ts | 4 - .../src/template/template-compilation.ts | 2 - .../src/capability/recall-definitions.ts | 2 - .../recall/src/capability/recall-function.ts | 2 - .../src/document/document-parsing.test.ts | 27 +- .../recall/src/document/document-parsing.ts | 3 +- .../recall/src/document/front-matter.ts | 8 +- .../recall/src/document/recall-document.ts | 1 - .../src/overview/recall-overview.test.ts | 181 ----------- .../recall/src/overview/recall-overview.ts | 119 ------- .../reference-common-case.overview.md | 38 --- .../overview/reference-example.overview.md | 43 --- docs/assets/openapi.json | 46 +-- packages/api/src/mcp/instructions.test.ts | 12 +- packages/api/src/mcp/instructions.ts | 2 +- .../src/mcp/tool-calling-instructions.test.ts | 4 +- packages/definitions/measure/measure-loads.ts | 24 +- .../definitions/src/capability/capability.ts | 36 +-- .../src/checking/schema-refusals.test.ts | 2 +- .../src/checking/schema-refusals.ts | 11 +- packages/definitions/src/document.ts | 3 +- .../src/document/front-matter-keys.ts | 18 +- .../src/document/front-matter-reading.ts | 16 +- .../src/document/front-matter.test.ts | 64 +--- .../src/document/function-fields.test.ts | 16 +- .../src/document/function-fields.ts | 4 - .../src/document/written-texts.test.ts | 192 ----------- .../definitions/src/document/written-texts.ts | 125 -------- .../src/events/playbook-facts.test.ts | 2 - packages/definitions/src/index.ts | 82 +---- .../definitions/src/markdown/html-blocks.ts | 56 ---- .../src/markdown/markdown-lines.ts | 298 ----------------- .../definitions/src/markdown/table-rows.ts | 20 -- .../src/markdown/written-blocks.test.ts | 163 ---------- .../src/operations/create-definition.test.ts | 47 ++- .../src/operations/create-definition.ts | 24 +- .../src/operations/definition-fields.ts | 21 +- .../src/operations/definition-reading.ts | 39 +-- .../src/operations/definition-views.ts | 31 +- .../src/operations/get-definition.test.ts | 13 +- .../src/operations/get-definition.ts | 2 +- .../src/operations/registry-access.ts | 8 +- .../src/operations/retire-definition.test.ts | 3 +- .../src/operations/retire-definition.ts | 8 +- .../src/operations/update-definition.test.ts | 51 +-- .../src/operations/update-definition.ts | 26 +- .../src/operations/used-by-overview.test.ts | 178 ----------- .../src/overview-words/call-words.test.ts | 62 ---- .../src/overview-words/call-words.ts | 70 ---- .../src/overview-words/called-words.test.ts | 105 ------ .../src/overview-words/called-words.ts | 28 -- .../src/overview-words/caller-words.test.ts | 80 ----- .../src/overview-words/caller-words.ts | 38 --- .../overview-words/computation-words.test.ts | 49 --- .../src/overview-words/computation-words.ts | 27 -- .../src/overview-words/diagram-labels.test.ts | 18 -- .../src/overview-words/diagram-labels.ts | 11 - .../overview-words/exact-name-words.test.ts | 27 -- .../src/overview-words/exact-name-words.ts | 8 - .../src/overview-words/field-meanings.test.ts | 102 ------ .../src/overview-words/field-meanings.ts | 116 ------- .../overview-words/field-sentences.test.ts | 187 ----------- .../src/overview-words/field-sentences.ts | 155 --------- .../reply-diagram-words.test.ts | 79 ----- .../src/overview-words/reply-diagram-words.ts | 49 --- .../src/overview-words/request-words.test.ts | 167 ---------- .../src/overview-words/request-words.ts | 155 --------- .../src/overview-words/source-words.test.ts | 25 -- .../src/overview-words/source-words.ts | 12 - .../src/overview-words/template-words.test.ts | 43 --- .../src/overview-words/template-words.ts | 15 - .../src/overview-words/tool-words.test.ts | 27 -- .../src/overview-words/tool-words.ts | 11 - .../src/overview-words/value-words.test.ts | 24 -- .../src/overview-words/value-words.ts | 14 - .../src/overview/definition-overview.test.ts | 266 ---------------- .../src/overview/definition-overview.ts | 96 ------ .../src/overview/duration-words.test.ts | 51 --- .../src/overview/duration-words.ts | 45 --- .../src/overview/event-filter-words.test.ts | 72 ----- .../src/overview/event-filter-words.ts | 121 ------- .../src/overview/inline-constructs.test.ts | 173 ---------- .../src/overview/inline-constructs.ts | 282 ----------------- .../src/overview/liquid-phrases.ts | 216 ------------- .../src/overview/liquid-reading.test.ts | 262 --------------- .../src/overview/liquid-reading.ts | 196 ------------ .../definitions/src/overview/markdown-text.ts | 208 ------------ .../src/overview/markdown-writer.test.ts | 211 ------------ .../src/overview/markdown-writer.ts | 149 --------- .../src/overview/mermaid-diagram.ts | 91 ------ .../src/overview/overview-blocks.test.ts | 233 -------------- .../src/overview/overview-parts.test.ts | 98 ------ packages/definitions/src/overview/overview.ts | 204 ------------ .../src/overview/quote-blocks.test.ts | 153 --------- .../src/overview/quoted-templates.test.ts | 74 ----- .../src/overview/schema-words.test.ts | 299 ------------------ .../src/plain-language/characters.test.ts | 3 +- .../plain-language/definition-words.test.ts | 2 +- .../src/plain-language/in-words.test.ts | 11 - .../src/plain-language/in-words.ts | 6 +- .../presenting/definition-presenter.test.ts | 22 +- .../src/presenting/definition-presenter.ts | 3 +- .../src/presenting/presenter-catalog.test.ts | 4 +- .../src/registry/definition-changes.test.ts | 6 +- .../src/registry/definition-events.ts | 2 +- .../src/registry/definition-versions.test.ts | 11 +- .../src/registry/definition.test.ts | 2 +- .../definitions/src/registry/definition.ts | 19 +- .../src/registry/definitions-decider.test.ts | 41 ++- .../src/registry/registry-decisions.ts | 15 +- .../schema-bounds.test.ts | 109 +++---- .../schema-bounds.ts | 0 .../schema-nouns.test.ts | 26 +- .../schema-nouns.ts | 6 +- .../schema-reading.ts | 0 .../src/schema-words/schema-words.test.ts | 234 ++++++++++++++ .../schema-words.ts | 149 +++------ .../src/shapes/categories-shape.ts | 2 +- .../definitions/src/shapes/period-forms.ts | 2 +- .../definitions/src/shapes/series-shape.ts | 2 +- .../src/shapes/shape-words.test.ts | 72 ----- .../definitions/src/shapes/shape-words.ts | 137 -------- .../definitions/src/shapes/table-shape.ts | 2 +- packages/definitions/src/shapes/text-shape.ts | 2 +- packages/definitions/src/template.ts | 8 - .../src/template/template-parsing.test.ts | 2 +- .../src/testing/capabilities/probe.ts | 11 - packages/definitions/src/testing/index.ts | 1 - .../src/testing/overview-sections.ts | 55 ---- .../src/used-by/definition-users.test.ts | 42 +-- .../src/used-by/definition-users.ts | 47 +-- .../src/used-by/overview-at-read.test.ts | 173 ---------- .../src/used-by/overview-at-read.ts | 72 ----- .../definitions/src/used-by/user-relations.ts | 14 +- packages/server/npm/imported-packages.test.ts | 4 +- .../computation/computation-over-http.test.ts | 2 +- .../server/src/mcp/served-schemas.test.ts | 4 +- packages/server/src/mcp/served-tools.test.ts | 12 +- .../src/reading/definition-overviews.test.ts | 208 ------------ .../reasoning/definitions-over-http.test.ts | 2 +- packages/tool-servers/src/index.ts | 7 - packages/tool-servers/src/policy.ts | 2 - packages/workflow-engine/src/index.ts | 10 +- .../workflow-host/measure/measured-views.ts | 3 +- .../folder-testing/projected-folders.test.ts | 7 +- .../src/folder-testing/tally-harness.ts | 3 +- .../projector/playbook-definitions.test.ts | 10 +- .../src/reaction-testing/playbook-writes.ts | 5 +- .../src/triggers/definition-records.test.ts | 9 +- .../src/views-testing/playbook-appends.ts | 2 - 238 files changed, 760 insertions(+), 12618 deletions(-) delete mode 100644 capabilities/computation/src/overview/computation-overview.test.ts delete mode 100644 capabilities/computation/src/overview/computation-overview.ts delete mode 100644 capabilities/computation/src/overview/month-window.overview.md delete mode 100644 capabilities/computation/src/overview/reference-example.overview.md delete mode 100644 capabilities/computation/src/overview/shaped-example.overview.md delete mode 100644 capabilities/coordination/src/document/workflow-texts.ts delete mode 100644 capabilities/coordination/src/overview/cron-words.ts delete mode 100644 capabilities/coordination/src/overview/prediction-example.overview.md delete mode 100644 capabilities/coordination/src/overview/reference-example.overview.md delete mode 100644 capabilities/coordination/src/overview/shaped-example.overview.md delete mode 100644 capabilities/coordination/src/overview/trigger-words.test.ts delete mode 100644 capabilities/coordination/src/overview/trigger-words.ts delete mode 100644 capabilities/coordination/src/overview/workflow-overview.test.ts delete mode 100644 capabilities/coordination/src/overview/workflow-overview.ts delete mode 100644 capabilities/coordination/src/steps/flow-words.ts delete mode 100644 capabilities/coordination/src/steps/grouped-tasks.test.ts delete mode 100644 capabilities/coordination/src/steps/incomplete-tasks.test.ts delete mode 100644 capabilities/coordination/src/steps/leaf-tasks.test.ts delete mode 100644 capabilities/coordination/src/steps/placed-tasks.ts delete mode 100644 capabilities/coordination/src/steps/task-diagram.ts delete mode 100644 capabilities/coordination/src/steps/task-edges.ts delete mode 100644 capabilities/coordination/src/steps/task-flow.test.ts delete mode 100644 capabilities/coordination/src/steps/task-flow.ts delete mode 100644 capabilities/coordination/src/steps/task-labels.ts delete mode 100644 capabilities/coordination/src/steps/task-reading.ts delete mode 100644 capabilities/coordination/src/steps/task-words.ts delete mode 100644 capabilities/coordination/src/steps/tried-tasks.test.ts delete mode 100644 capabilities/coordination/src/testing/done-steps.ts delete mode 100644 capabilities/interaction/src/overview/approve-report.overview.md delete mode 100644 capabilities/interaction/src/overview/call-overview.ts delete mode 100644 capabilities/interaction/src/overview/interaction-overview.test.ts delete mode 100644 capabilities/interaction/src/overview/reference-call.overview.md delete mode 100644 capabilities/interaction/src/overview/reference-chat.overview.md delete mode 100644 capabilities/interaction/src/overview/reference-inbox.overview.md delete mode 100644 capabilities/interaction/src/overview/request-overview.test.ts delete mode 100644 capabilities/interaction/src/overview/request-overview.ts delete mode 100644 capabilities/interaction/src/overview/request-party.ts delete mode 100644 capabilities/prediction/src/overview/data-words.ts delete mode 100644 capabilities/prediction/src/overview/evaluation-words.ts delete mode 100644 capabilities/prediction/src/overview/prediction-overview.test.ts delete mode 100644 capabilities/prediction/src/overview/prediction-overview.ts delete mode 100644 capabilities/prediction/src/overview/predicts-words.ts delete mode 100644 capabilities/prediction/src/overview/reference-example.overview.md delete mode 100644 capabilities/prediction/src/overview/reference-uploaded.overview.md delete mode 100644 capabilities/prediction/src/overview/trainer-words.ts delete mode 100644 capabilities/reasoning/src/overview/reasoning-overview.test.ts delete mode 100644 capabilities/reasoning/src/overview/reasoning-overview.ts delete mode 100644 capabilities/reasoning/src/overview/reference-example.overview.md delete mode 100644 capabilities/reasoning/src/overview/reference-tools.overview.md delete mode 100644 capabilities/reasoning/src/overview/shaped-example.overview.md delete mode 100644 capabilities/recall/src/overview/recall-overview.test.ts delete mode 100644 capabilities/recall/src/overview/recall-overview.ts delete mode 100644 capabilities/recall/src/overview/reference-common-case.overview.md delete mode 100644 capabilities/recall/src/overview/reference-example.overview.md delete mode 100644 packages/definitions/src/document/written-texts.test.ts delete mode 100644 packages/definitions/src/document/written-texts.ts delete mode 100644 packages/definitions/src/markdown/html-blocks.ts delete mode 100644 packages/definitions/src/markdown/markdown-lines.ts delete mode 100644 packages/definitions/src/markdown/table-rows.ts delete mode 100644 packages/definitions/src/markdown/written-blocks.test.ts delete mode 100644 packages/definitions/src/operations/used-by-overview.test.ts delete mode 100644 packages/definitions/src/overview-words/call-words.test.ts delete mode 100644 packages/definitions/src/overview-words/call-words.ts delete mode 100644 packages/definitions/src/overview-words/called-words.test.ts delete mode 100644 packages/definitions/src/overview-words/called-words.ts delete mode 100644 packages/definitions/src/overview-words/caller-words.test.ts delete mode 100644 packages/definitions/src/overview-words/caller-words.ts delete mode 100644 packages/definitions/src/overview-words/computation-words.test.ts delete mode 100644 packages/definitions/src/overview-words/computation-words.ts delete mode 100644 packages/definitions/src/overview-words/diagram-labels.test.ts delete mode 100644 packages/definitions/src/overview-words/diagram-labels.ts delete mode 100644 packages/definitions/src/overview-words/exact-name-words.test.ts delete mode 100644 packages/definitions/src/overview-words/exact-name-words.ts delete mode 100644 packages/definitions/src/overview-words/field-meanings.test.ts delete mode 100644 packages/definitions/src/overview-words/field-meanings.ts delete mode 100644 packages/definitions/src/overview-words/field-sentences.test.ts delete mode 100644 packages/definitions/src/overview-words/field-sentences.ts delete mode 100644 packages/definitions/src/overview-words/reply-diagram-words.test.ts delete mode 100644 packages/definitions/src/overview-words/reply-diagram-words.ts delete mode 100644 packages/definitions/src/overview-words/request-words.test.ts delete mode 100644 packages/definitions/src/overview-words/request-words.ts delete mode 100644 packages/definitions/src/overview-words/source-words.test.ts delete mode 100644 packages/definitions/src/overview-words/source-words.ts delete mode 100644 packages/definitions/src/overview-words/template-words.test.ts delete mode 100644 packages/definitions/src/overview-words/template-words.ts delete mode 100644 packages/definitions/src/overview-words/tool-words.test.ts delete mode 100644 packages/definitions/src/overview-words/tool-words.ts delete mode 100644 packages/definitions/src/overview-words/value-words.test.ts delete mode 100644 packages/definitions/src/overview-words/value-words.ts delete mode 100644 packages/definitions/src/overview/definition-overview.test.ts delete mode 100644 packages/definitions/src/overview/definition-overview.ts delete mode 100644 packages/definitions/src/overview/duration-words.test.ts delete mode 100644 packages/definitions/src/overview/duration-words.ts delete mode 100644 packages/definitions/src/overview/event-filter-words.test.ts delete mode 100644 packages/definitions/src/overview/event-filter-words.ts delete mode 100644 packages/definitions/src/overview/inline-constructs.test.ts delete mode 100644 packages/definitions/src/overview/inline-constructs.ts delete mode 100644 packages/definitions/src/overview/liquid-phrases.ts delete mode 100644 packages/definitions/src/overview/liquid-reading.test.ts delete mode 100644 packages/definitions/src/overview/liquid-reading.ts delete mode 100644 packages/definitions/src/overview/markdown-text.ts delete mode 100644 packages/definitions/src/overview/markdown-writer.test.ts delete mode 100644 packages/definitions/src/overview/markdown-writer.ts delete mode 100644 packages/definitions/src/overview/mermaid-diagram.ts delete mode 100644 packages/definitions/src/overview/overview-blocks.test.ts delete mode 100644 packages/definitions/src/overview/overview-parts.test.ts delete mode 100644 packages/definitions/src/overview/overview.ts delete mode 100644 packages/definitions/src/overview/quote-blocks.test.ts delete mode 100644 packages/definitions/src/overview/quoted-templates.test.ts delete mode 100644 packages/definitions/src/overview/schema-words.test.ts rename packages/definitions/src/{overview => schema-words}/schema-bounds.test.ts (66%) rename packages/definitions/src/{overview => schema-words}/schema-bounds.ts (100%) rename packages/definitions/src/{overview => schema-words}/schema-nouns.test.ts (63%) rename packages/definitions/src/{overview => schema-words}/schema-nouns.ts (93%) rename packages/definitions/src/{overview => schema-words}/schema-reading.ts (100%) create mode 100644 packages/definitions/src/schema-words/schema-words.test.ts rename packages/definitions/src/{overview => schema-words}/schema-words.ts (59%) delete mode 100644 packages/definitions/src/shapes/shape-words.test.ts delete mode 100644 packages/definitions/src/shapes/shape-words.ts delete mode 100644 packages/definitions/src/testing/overview-sections.ts delete mode 100644 packages/definitions/src/used-by/overview-at-read.test.ts delete mode 100644 packages/definitions/src/used-by/overview-at-read.ts delete mode 100644 packages/server/src/reading/definition-overviews.test.ts diff --git a/.oxfmtrc.json b/.oxfmtrc.json index 8db13a53b..23edc3dd2 100644 --- a/.oxfmtrc.json +++ b/.oxfmtrc.json @@ -12,8 +12,7 @@ "pnpm-lock.yaml", "CHANGELOG.md", "packages/api/src/testing/openapi/openapi-3.1-schema-2022-10-07.json", - "docs/assets/openapi.json", - "**/*.overview.md" + "docs/assets/openapi.json" ], "overrides": [ { diff --git a/capabilities/computation/src/capability/computation-function.test.ts b/capabilities/computation/src/capability/computation-function.test.ts index 24ca717f4..50316f0c7 100644 --- a/capabilities/computation/src/capability/computation-function.test.ts +++ b/capabilities/computation/src/capability/computation-function.test.ts @@ -166,7 +166,7 @@ describe('a computation function definition that is refused', () => { { pointer: '', detail: - 'Line 3, /model: model is not a key of the front matter; it takes description, about, language, input, output', + 'Line 3, /model: model is not a key of the front matter; it takes description, language, input, output', }, { pointer: '', detail: 'Line 5: The definition has no program: write it after the front matter' }, ], diff --git a/capabilities/computation/src/capability/computation-function.ts b/capabilities/computation/src/capability/computation-function.ts index 5b4ff7195..3aa10479b 100644 --- a/capabilities/computation/src/capability/computation-function.ts +++ b/capabilities/computation/src/capability/computation-function.ts @@ -14,7 +14,6 @@ import { Effect, Result } from 'effect'; import type { ComputationFunctionDefinitionDocument } from '../document/computation-document.ts'; import { documentCheck, invalidDefinition } from '../document/document-check.ts'; import { parseComputationDocument } from '../document/document-parsing.ts'; -import { computationOverview } from '../overview/computation-overview.ts'; import { computationRun } from '../run/computation-run.ts'; import { computationBounds } from '../run/run-bounds.ts'; @@ -32,10 +31,9 @@ function parse(source: string): Effect.Effect run(document, input, stripped), longestAnyRunMs: deadlineMs, }); diff --git a/capabilities/computation/src/document/computation-document.ts b/capabilities/computation/src/document/computation-document.ts index d23fb743c..e07c3859b 100644 --- a/capabilities/computation/src/document/computation-document.ts +++ b/capabilities/computation/src/document/computation-document.ts @@ -7,7 +7,6 @@ export interface OutputContract extends ValueContract { export interface ComputationFunctionDefinitionDocument { readonly description?: string; - readonly about?: string; readonly language: 'typescript'; readonly input: ValueContract; readonly output: OutputContract; diff --git a/capabilities/computation/src/document/document-parsing.test.ts b/capabilities/computation/src/document/document-parsing.test.ts index 3ae11fa97..d9b4086af 100644 --- a/capabilities/computation/src/document/document-parsing.test.ts +++ b/capabilities/computation/src/document/document-parsing.test.ts @@ -91,9 +91,9 @@ describe('the front matter of a computation function definition', () => { ), ), ).toEqual([ - 'Line 3, /model: model is not a key of the front matter; it takes description, about, language, input, output', - 'Line 4, /config: config is not a key of the front matter; it takes description, about, language, input, output', - 'Line 5, /tools: tools is not a key of the front matter; it takes description, about, language, input, output', + 'Line 3, /model: model is not a key of the front matter; it takes description, language, input, output', + 'Line 4, /config: config is not a key of the front matter; it takes description, language, input, output', + 'Line 5, /tools: tools is not a key of the front matter; it takes description, language, input, output', 'Line 6, /input/default: default is not a key of input; it takes schema', 'Line 7, /output/format: format is not a key of output; it takes schema, shape', ]); @@ -118,7 +118,7 @@ describe('the values in the front matter of a computation function definition', it('is checked with the program, whose issues it reports beside its own', () => { expect(issuesIn(programDocument(' ', 'language: typescript\nsize: 2'))).toEqual([ - 'Line 3, /size: size is not a key of the front matter; it takes description, about, language, input, output', + 'Line 3, /size: size is not a key of the front matter; it takes description, language, input, output', 'Line 5: The definition has no program: write it after the front matter', ]); expect(issuesIn('---\nlanguage: [typescript\n---\n')).toEqual([ @@ -141,23 +141,3 @@ describe('the program of a computation function definition', () => { ).toBe('export default function (input: Input): Output {\n return input.perod;\n}'); }); }); - -describe('the about of a computation function definition', () => { - it('is kept as its author wrote it', () => { - const frontMatter = 'language: typescript\nabout: |\n The finance team reads it.'; - - expect(parsed(programDocument(functionOf('return input;'), frontMatter)).about).toBe( - 'The finance team reads it.\n', - ); - }); - - it('takes no heading, nor does the description, each refused at its line', () => { - const frontMatter = - "language: typescript\ndescription: '# Adds one'\nabout: |\n The finance team reads it.\n ---"; - - expect(issuesIn(programDocument(functionOf('return input;'), frontMatter))).toEqual([ - 'Line 3, /description: description takes no heading, since an overview writes its own: the line that begins “# Adds one” makes one, with # before its words or as a line of = or - under the words above it', - 'Line 6, /about: about takes no heading, since an overview writes its own: this line makes one, with # before its words or as a line of = or - under the words above it', - ]); - }); -}); diff --git a/capabilities/computation/src/document/document-parsing.ts b/capabilities/computation/src/document/document-parsing.ts index 51cfab18d..3d70d2acc 100644 --- a/capabilities/computation/src/document/document-parsing.ts +++ b/capabilities/computation/src/document/document-parsing.ts @@ -86,7 +86,7 @@ function documentFrom( if (issues.length > 0 || Result.isFailure(decoded)) { return Result.fail([...issues, ...issuesOf(program)]); } - const { description, about, input, output } = decoded.success; + const { description, input, output } = decoded.success; const parts = { language: languageOf(decoded.success.language, lines), input: contractOf(input, 'input', lines), @@ -103,7 +103,6 @@ function documentFrom( ? Result.fail(found) : Result.map(Result.all(parts), (checked) => ({ ...(description === undefined ? {} : { description }), - ...(about === undefined ? {} : { about }), language: checked.language, input: checked.input, output: checked.output, diff --git a/capabilities/computation/src/document/front-matter.ts b/capabilities/computation/src/document/front-matter.ts index 53f6b36a2..c3988ad1c 100644 --- a/capabilities/computation/src/document/front-matter.ts +++ b/capabilities/computation/src/document/front-matter.ts @@ -1,9 +1,4 @@ -import { - AboutSchema, - DescriptionSchema, - type FrontMatterSection, - type FrontMatterShape, -} from '@on.auto/definitions/document'; +import { DescriptionSchema, type FrontMatterSection, type FrontMatterShape } from '@on.auto/definitions/document'; import { Schema } from 'effect'; const SectionSchema = Schema.Struct({ schema: Schema.optionalKey(Schema.JsonObject) }); @@ -15,7 +10,6 @@ const OutputSectionSchema = Schema.Struct({ const FrontMatterSchema = Schema.Struct({ description: Schema.optionalKey(DescriptionSchema), - about: Schema.optionalKey(AboutSchema), language: Schema.String, input: Schema.optionalKey(SectionSchema), output: Schema.optionalKey(OutputSectionSchema), diff --git a/capabilities/computation/src/overview/computation-overview.test.ts b/capabilities/computation/src/overview/computation-overview.test.ts deleted file mode 100644 index 2a706ae48..000000000 --- a/capabilities/computation/src/overview/computation-overview.test.ts +++ /dev/null @@ -1,145 +0,0 @@ -import { readFileSync } from 'node:fs'; - -import { markdownOf } from '@on.auto/definitions'; -import { overviewAsRead } from '@on.auto/definitions/testing'; -import { describe, expect, it } from 'vitest'; - -import { computationWith } from '../testing/computation-runs.ts'; - -const page = readFileSync(new URL('../../../../docs/reference/computation-format.md', import.meta.url), 'utf8'); - -const golden = readFileSync(new URL('reference-example.overview.md', import.meta.url), 'utf8'); - -const [, example = ''] = /```markdown\n([\s\S]*?)```/u.exec(page) ?? []; - -const shapesPage = readFileSync(new URL('../../../../docs/reference/output-shapes.md', import.meta.url), 'utf8'); - -const [, shapedExample = ''] = [...shapesPage.matchAll(/```markdown\n([\s\S]*?)```/gu)][1] ?? []; - -const monthWindow = [ - '---', - `description: 'Turns a month such as 2026-04 or "April 2026" into its date range, the previous month''s date range and readable labels'`, - 'language: typescript', - 'input:', - ' schema:', - ' type: object', - ' required: [month]', - ' properties:', - " month: { type: string, minLength: 6, maxLength: 20, description: 'The month to report on, as 2026-04 or April 2026' }", - 'output:', - ' schema:', - ' type: object', - ' required: [month, label, start, end, prev_label, prev_start, prev_end]', - ' properties:', - ' month: { type: string }', - ' label: { type: string }', - ' start: { type: string }', - ' end: { type: string }', - ' prev_label: { type: string }', - ' prev_start: { type: string }', - ' prev_end: { type: string }', - '---', - 'export default function (input: Input): Output {', - " const [year = '', month = ''] = input.month.split('-');", - " const previous = month === '01' ? `${Number(year) - 1}-12` : `${year}-${String(Number(month) - 1).padStart(2, '0')}`;", - ' return {', - ' month: input.month,', - ' label: input.month,', - ' start: `${input.month}-01`,', - ' end: `${input.month}-28`,', - ' prev_label: previous,', - ' prev_start: `${previous}-01`,', - ' prev_end: `${previous}-28`,', - ' };', - '}', -].join('\n'); - -function overviewOf(source: string, name: string): string { - return markdownOf(computationWith().prepared(source).overview(name)); -} - -describe('the overview of the example of the reference of computation functions', () => { - it('is, byte for byte, the overview pinned beside this test', () => { - expect(overviewOf(example, 'campaign-pace')).toBe(golden); - }); -}); - -describe('the overview of the computation function of the reference of output shapes', () => { - it('is, byte for byte, the overview pinned beside this test, the shape it answers told in its sentence', () => { - expect(overviewOf(shapedExample, 'monthly-sign-ups')).toBe( - readFileSync(new URL('shaped-example.overview.md', import.meta.url), 'utf8'), - ); - }); -}); - -describe('the overview of a computation function a workflow calls', () => { - it('tells what it is given and answers by the meanings of its fields, and is, as read, the overview pinned beside this test', () => { - const overview = computationWith().prepared(monthWindow).overview('month-window'); - const caller = { - type: 'workflow', - name: 'monthly-website-report', - details: { - title: 'Monthly website report', - calls: [{ type: 'computation', name: 'month-window', steps: [{ number: 1, name: 'work-out-month' }] }], - }, - }; - - expect(overviewAsRead({ type: 'computation', name: 'month-window', overview }, [caller])).toBe( - readFileSync(new URL('month-window.overview.md', import.meta.url), 'utf8'), - ); - }); -}); - -describe('the overview of a computation function', () => { - it('says that a function without schemas is given any input and answers any value, and holds the rest under Details', () => { - const source = [ - '---', - 'language: typescript', - '---', - 'export default function (input: Input): Output {', - ' return input;', - '}', - ]; - - expect(overviewOf(source.join('\n'), 'echo')).toBe( - [ - '# echo', - 'This computation function has no description yet.', - 'It is given any input, and a program works out the answer, the same every time for the same input: any value.', - '## Details', - '### Input', - 'It takes any input, which its document does not describe.', - '### Output', - 'It produces any output, which its document does not describe.', - '### Program', - "TypeScript, in the definition's document.\n", - ].join('\n\n'), - ); - }); -}); - -describe('the overview of a computation function whose document has an about', () => { - it('writes the about as its author wrote it, after the description and before what it is given', () => { - const source = [ - '---', - 'description: Adds one', - 'about: |', - ' The finance team reads it to see *how many* campaigns there are.', - '', - ' A workflow runs it at the end of each period.', - 'language: typescript', - '---', - 'export default function (input: Input): Output {', - ' return input;', - '}', - ]; - - expect(overviewOf(source.join('\n'), 'count').split('\n\n').slice(0, 5)).toEqual([ - '# count', - 'Adds one.', - 'The finance team reads it to see *how many* campaigns there are.', - 'A workflow runs it at the end of each period.', - 'It is given any input, and a program works out the answer, the same every time for the same input: any value.', - ]); - }); -}); diff --git a/capabilities/computation/src/overview/computation-overview.ts b/capabilities/computation/src/overview/computation-overview.ts deleted file mode 100644 index 89f59b798..000000000 --- a/capabilities/computation/src/overview/computation-overview.ts +++ /dev/null @@ -1,14 +0,0 @@ -import { computedSentence, inputDetails, outputDetails, programSection, type ToldParts } from '@on.auto/definitions'; - -import type { ComputationFunctionDefinitionDocument } from '../document/computation-document.ts'; - -export function computationOverview({ language, input, output }: ComputationFunctionDefinitionDocument): ToldParts { - return { - body: [computedSentence({ input: input.schema?.document, output: output.schema?.document, shape: output.shape })], - appendix: [ - inputDetails(input.schema?.document), - outputDetails(output.schema?.document, output.shape), - programSection(language), - ], - }; -} diff --git a/capabilities/computation/src/overview/month-window.overview.md b/capabilities/computation/src/overview/month-window.overview.md deleted file mode 100644 index acd9174a0..000000000 --- a/capabilities/computation/src/overview/month-window.overview.md +++ /dev/null @@ -1,37 +0,0 @@ -# month-window - -Turns a month such as 2026-04 or "April 2026" into its date range, the previous month's date range and readable labels. - -It is given the month to report on, as 2026-04 or April 2026, and a program works out the answer, the same every time for the same input: the month, the label, the start, the end, the prev label, the prev start and the prev end. - -## Used by - -- The workflow **Monthly website report** calls it in step 1, **Work out month**. - -## Details - -### Input - -| Field | Required | What it is | -| --- | --- | --- | -| **month** | yes | The month to report on, as 2026-04 or April 2026. Text, 6 to 20 characters. | - -### Output - -| Field | Required | What it is | -| --- | --- | --- | -| **month** | yes | Text. | -| **label** | yes | Text. | -| **start** | yes | Text. | -| **end** | yes | Text. | -| **prev_label** | yes | Text. | -| **prev_start** | yes | Text. | -| **prev_end** | yes | Text. | - -### Program - -TypeScript, in the definition's document. - -### Used by - -- `monthly-website-report`, a workflow diff --git a/capabilities/computation/src/overview/reference-example.overview.md b/capabilities/computation/src/overview/reference-example.overview.md deleted file mode 100644 index 25b59c19d..000000000 --- a/capabilities/computation/src/overview/reference-example.overview.md +++ /dev/null @@ -1,25 +0,0 @@ -# campaign-pace - -Spend, pace and projection per campaign, in cents, for a reporting period. - -It is given the rows and the period, and a program works out the answer, the same every time for the same input: the campaigns and the total spend cents. - -## Details - -### Input - -| Field | Required | What it is | -| --- | --- | --- | -| **rows** | yes | A list of at most 4,000 records with: **campaign** (text), **cost_cents** (a whole number, from 0 to 5,000,000), **budget_cents** (a whole number, from 0 to 1,000,000,000). | -| **period** | yes | A record with: **days_elapsed** (a whole number, from 1 to 366), **days_total** (a whole number, from 1 to 366). | - -### Output - -| Field | Required | What it is | -| --- | --- | --- | -| **campaigns** | yes | A list of records. | -| **total_spend_cents** | yes | A whole number. | - -### Program - -TypeScript, in the definition's document. diff --git a/capabilities/computation/src/overview/shaped-example.overview.md b/capabilities/computation/src/overview/shaped-example.overview.md deleted file mode 100644 index b569c38b7..000000000 --- a/capabilities/computation/src/overview/shaped-example.overview.md +++ /dev/null @@ -1,21 +0,0 @@ -# monthly-sign-ups - -Monthly sign-ups, with their monthly average marked. - -It is given the months, and a program works out the answer, the same every time for the same input: a series over time. - -## Details - -### Input - -| Field | Required | What it is | -| --- | --- | --- | -| **months** | yes | A list of 1 to 120 records with: **month** (text), **signups** (a whole number, at least 0). | - -### Output - -It answers a series over time, in **unit**: 1 to 20 lines, one for each **name**, of at most 2,000 **points** in all, each a **value** at its **at**, with a **low** and a **high** where given, and at most 8 **marks**, each a **value** with a **label**. Each **at** is a day, week, month, quarter or year, as **period** names, or a date and time where it names none, and **origin** is the last one observed. - -### Program - -TypeScript, in the definition's document. diff --git a/capabilities/coordination/src/capability/workflow.ts b/capabilities/coordination/src/capability/workflow.ts index dbc417f10..770a08717 100644 --- a/capabilities/coordination/src/capability/workflow.ts +++ b/capabilities/coordination/src/capability/workflow.ts @@ -13,7 +13,6 @@ import { Effect, type Schema } from 'effect'; import { checkedExpressions, type ExpressionCheck } from '../document/expression-check.ts'; import { runnableDocument, workflowReaderOf, type ReadWorkflow } from '../document/workflow-document.ts'; import { summaryOf } from '../document/workflow-summary.ts'; -import { workflowOverview } from '../overview/workflow-overview.ts'; import { runPresenter } from '../presenting/run-presenter.ts'; import { started, type WorkflowStarts } from '../runs/run-start.ts'; @@ -47,7 +46,6 @@ export function makeWorkflowAdapter(dependencies: WorkflowAdapterDependencies): readWorkflowDocument(source, dependencies.mostDurationMs), check: checkedExpressions(dependencies.check), summarize: ({ document }) => summaryOf(document), - overview: workflowOverview, run: (read, input, run, stripped) => Effect.flatMap(admittedInput(read, input, run), (admitted) => started(dependencies, runnableDocument(read.document, stripped), admitted, run), diff --git a/capabilities/coordination/src/document/workflow-document.test.ts b/capabilities/coordination/src/document/workflow-document.test.ts index a190f22de..f242daf5f 100644 --- a/capabilities/coordination/src/document/workflow-document.test.ts +++ b/capabilities/coordination/src/document/workflow-document.test.ts @@ -123,13 +123,7 @@ describe('a workflow document that nests too deeply', () => { }); describe('the summary of a workflow definition', () => { - it('carries the about of the metadata of the document, which its overview writes', () => { - expect( - summaryOf({ document: { summary: 'Greets', metadata: { about: 'It serves the front desk.' } }, do: [] }), - ).toEqual({ description: 'Greets', about: 'It serves the front desk.' }); - }); - - it('is the summary or title of the document, its inline input and output schemas, and its title in its details', () => { + it('is the summary or title of the document and its inline input and output schemas', () => { expect( summaryOf({ document: { summary: 'Greets', title: 'Greeting' }, @@ -141,12 +135,8 @@ describe('the summary of a workflow definition', () => { description: 'Greets', inputSchema: { type: 'object' }, outputSchema: { type: 'string' }, - details: { title: 'Greeting' }, - }); - expect(summaryOf({ document: { title: 'Greeting' }, do: [] })).toEqual({ - description: 'Greeting', - details: { title: 'Greeting' }, }); + expect(summaryOf({ document: { title: 'Greeting' }, do: [] })).toEqual({ description: 'Greeting' }); expect(summaryOf({ do: [] })).toEqual({}); }); @@ -159,65 +149,3 @@ describe('the summary of a workflow definition', () => { }); }); }); - -const greeting = 'do:\n - greet: { set: { a: 1 } }\n'; - -const notRun = 'The workflow document is not a workflow this runtime runs'; - -function withAbout(about: string): string { - return `${header} metadata:\n about: ${about}\n${greeting}`; -} - -describe('the about of a workflow document', () => { - it('is text of 1 to 4000 characters under the metadata of the document, refused at its line otherwise', async () => { - expect(await parsed(withAbout('5'))).toMatchObject( - rejected(notRun, ['Line 7, column 12: at /document/metadata/about: Expected string']), - ); - expect(await parsed(withAbout('x'.repeat(4001)))).toMatchObject( - rejected(notRun, [ - 'Line 7, column 12: at /document/metadata/about: Expected a value with a length of at most 4000', - ]), - ); - expect(await parsed(withAbout('x'.repeat(4000)))).toMatchObject({ _tag: 'Success' }); - }); -}); - -describe('the summary, the title and the about of a workflow document', () => { - it('take no heading, each refused at its line in a literal block and at the line its value opens in any other style', async () => { - const about = [' about: |', ' It serves the front desk.', '', ' ## When it runs'].join('\n'); - const source = `${header} summary: '# Greets a guest'\n metadata:\n${about}\n${greeting}`; - - expect(await parsed(source)).toMatchObject( - rejected(notRun, [ - 'Line 6, column 12: at /document/summary: summary takes no heading, since an overview writes its own: the line that begins “# Greets a guest” makes one, with # before its words or as a line of = or - under the words above it', - 'Line 11, column 7: at /document/metadata/about: about takes no heading, since an overview writes its own: this line makes one, with # before its words or as a line of = or - under the words above it', - ]), - ); - }); - - it('refuse a heading in the title where it is the description, and take one where the summary is', async () => { - const titled = `${header} title: '# Greeting'\n`; - - expect(await parsed(`${titled}${greeting}`)).toMatchObject( - rejected(notRun, [ - 'Line 6, column 10: at /document/title: title takes no heading, since an overview writes its own: the line that begins “# Greeting” makes one, with # before its words or as a line of = or - under the words above it', - ]), - ); - expect(await parsed(`${titled} summary: Greets a guest\n${greeting}`)).toMatchObject({ _tag: 'Success' }); - }); - - it('leave no block open and hold no line of the overview, at the line of the text in a literal block or of the value', async () => { - const about = [' about: |', ' It serves the front desk.', ' n2 - n2 -.->|"on error 409"| n3 -``` - -Its steps, numbered as in the diagram: - -1. **forecast** tries step 2; on an error with status 409, when `$error.kind == "no_active_version"`, it does step 3. -2. **ask** runs the prediction function **weekly-orders**. -3. **none** sets **forecast**. - -## What it calls - -It calls the prediction function **Weekly orders**: Predicts the orders of each region one and four weeks ahead. - -## Details - -### What it calls - -- `weekly-orders`, a prediction function diff --git a/capabilities/coordination/src/overview/reference-example.overview.md b/capabilities/coordination/src/overview/reference-example.overview.md deleted file mode 100644 index 24d872bae..000000000 --- a/capabilities/coordination/src/overview/reference-example.overview.md +++ /dev/null @@ -1,59 +0,0 @@ -# review-and-approve - -Reviews a campaign brief, then asks its owner to approve the review. - -## What starts it - -- By a call of `run_definition`, from a caller or from a step of another workflow - -## What it needs - -| Field | Required | What it is | -| --- | --- | --- | -| **brief** | yes | Text. | -| **owner** | yes | Text. | - -## What it produces - -It produces any output, which its document does not describe. - -## What it does - -```mermaid -flowchart LR - subgraph g1["1. Try"] - n2["2. Run review-campaign-brief"] - end - subgraph g2["4. Try"] - n4["5. Run approve-campaign-brief"] - end - n1(["By a call"]) - n3(["3. Raise “The review could not be run”"]) - n5["6. Set verdict and note"] - n1 --> n2 - n2 --> n4 - n2 -.->|"on error 503, after retries"| n3 - n4 -.->|"on an error"| n5 -``` - -Its steps, numbered as in the diagram: - -1. **review** tries step 2; on an error with status 503, it tries again under the retry policy **patient**, and once no retry is left, it does step 3. -2. **review-brief** runs the reasoning function **review-campaign-brief**. -3. **give-up** raises the error *The review could not be run*, with status 503. -4. **approval** tries step 5; on an error whose type is `https://on.auto/problems/unanswered`, it does step 6. -5. **ask-for-approval** runs the interaction function **approve-campaign-brief**. -6. **no-answer** sets **verdict** and **note**. - -## What it calls - -It calls the reasoning function **Review campaign brief**: Reviews a campaign brief against four criteria and recommends Ready or Revise. - -It calls the interaction function **Approve campaign brief**: Asks the campaign owner to approve a reviewed brief. - -## Details - -### What it calls - -- `review-campaign-brief`, a reasoning function -- `approve-campaign-brief`, an interaction function diff --git a/capabilities/coordination/src/overview/shaped-example.overview.md b/capabilities/coordination/src/overview/shaped-example.overview.md deleted file mode 100644 index 71e42ad57..000000000 --- a/capabilities/coordination/src/overview/shaped-example.overview.md +++ /dev/null @@ -1,41 +0,0 @@ -# campaign-pace-table - -Computes the pace of each campaign and answers it as a table. - -## What starts it - -- By a call of `run_definition`, from a caller or from a step of another workflow - -## What it needs - -| Field | Required | What it is | -| --- | --- | --- | -| **rows** | yes | A list. | -| **period** | yes | A record. | - -## What it produces - -It answers a table of 1 to 50 **columns**, each a **name** of its own with a **unit** where given, and at most 500 **rows**, each one cell for each column: text, a number, yes or no, or nothing. - -## What it does - -```mermaid -flowchart LR - n1(["By a call"]) - n2["1. Run campaign-pace"] - n1 --> n2 -``` - -Its steps, numbered as in the diagram: - -1. **pace** runs the computation function **campaign-pace**. - -## What it calls - -It calls the computation function **Campaign pace**: Spend, pace and projection per campaign, in cents, for a reporting period. - -## Details - -### What it calls - -- `campaign-pace`, a computation function diff --git a/capabilities/coordination/src/overview/trigger-words.test.ts b/capabilities/coordination/src/overview/trigger-words.test.ts deleted file mode 100644 index 48f531922..000000000 --- a/capabilities/coordination/src/overview/trigger-words.test.ts +++ /dev/null @@ -1,124 +0,0 @@ -import { plainTextOf, type Phrases } from '@on.auto/definitions'; -import { describe, expect, it } from 'vitest'; - -import { summaryOf } from '../document/workflow-summary.ts'; -import { workflow } from '../testing/workflows.ts'; -import { cronInWords } from './cron-words.ts'; -import { startsOf } from './trigger-words.ts'; - -function startsIn(schedule: string) { - return startsOf(summaryOf(workflow(`schedule:\n${schedule}\ndo: []`)).triggers ?? []); -} - -const eventSchedule = [ - ' on:', - ' any:', - ' - with: { type: com.example.brief.submitted, data: \'${ $data.priority === "high" }\' }', - ' - with: { type: run_succeeded, subject: reasoning/summarize, depth: 2 }', - ' - with: { type: com.example.brief.closed, data: { closed: true } }', -].join('\n'); - -function plainOf(phrases: Phrases): string { - return phrases - .map((phrase) => { - if (typeof phrase === 'string') { - return phrase; - } - return 'name' in phrase ? `**${phrase.name}**` : `\`${plainTextOf([phrase])}\``; - }) - .join(''); -} - -describe('the words of an event trigger', () => { - it('say each of its filters in the words of an event filter, with what each tests, an event at a time', () => { - expect(startsIn(eventSchedule).map(({ label, words }) => [label, plainOf(words)])).toEqual([ - [ - 'On com.example.brief.submitted, run_succeeded, or com.example.brief.closed', - [ - 'On an event, each time an event `com.example.brief.submitted` is published to the playbook, when `$data.priority === "high"` holds for its data,', - 'each time a run of the reasoning function **summarize** succeeds, when its `depth` is `2`,', - 'or each time an event `com.example.brief.closed` is published to the playbook, when its data is exactly `{"closed":true}`', - ].join(' '), - ], - ['By a call', 'By a call of `run_definition`, from a caller or from a step of another workflow'], - ]); - }); -}); - -describe('the words of each way a workflow starts', () => { - it('say an event of a type alone, a cron schedule in words where it reads plainly, else as written, and an every', () => { - const [ping, plain, every] = startsIn( - " on: { one: { with: { type: com.example.ping } } }\n cron: '30 18 * * *'\n every: { hours: 1, minutes: 30 }", - ); - const [written] = startsIn(" cron: '*/5 * * * *'"); - - expect([ping, plain, every, written]).toEqual([ - { - label: 'On com.example.ping', - words: [ - 'On an event, ', - 'each time ', - 'an event ', - { code: 'com.example.ping' }, - ' is published to the playbook', - ], - }, - { label: 'At 18:30 UTC every day', words: ['On a schedule, at 18:30 UTC every day'] }, - { - label: 'Every 1 hour and 30 minutes', - words: ['Every 1 hour and 30 minutes, counted from when the trigger was saved'], - }, - { - label: 'On the schedule */5 * * * *', - words: ['On the cron schedule ', { code: '*/5 * * * *' }, ', read in UTC'], - }, - ]); - }); - - it('end with a call, the one way a workflow without a schedule starts', () => { - expect(startsOf([])).toEqual([ - { - label: 'By a call', - words: ['By a call of ', { code: 'run_definition' }, ', from a caller or from a step of another workflow'], - }, - ]); - }); -}); - -describe('a cron schedule in words', () => { - it('names the time in UTC and the days, a day of the week, a range, a list or a day of the month', () => { - expect( - ['0 9 * * 1-5', '0 7 * * 1', '0 7 * * 0', '0 7 * * 7', '0 7 * * 1,3,5', '0 0 1 * *', ' 5 23 * * * '].map((cron) => - cronInWords(cron), - ), - ).toEqual([ - 'at 09:00 UTC from Monday to Friday', - 'at 07:00 UTC every Monday', - 'at 07:00 UTC every Sunday', - 'at 07:00 UTC every Sunday', - 'at 07:00 UTC on Monday, Wednesday, and Friday', - 'at 00:00 UTC on day 1 of each month', - 'at 23:05 UTC every day', - ]); - }); - - it('reads no words of a schedule that does not read plainly', () => { - expect( - [ - '*/5 * * * *', - '0 */2 * * *', - '60 9 * * *', - '0 24 * * *', - '0 9 * 1 *', - '0 9 1 * 1', - '0 9 32 * *', - '0 9 * * 5-1', - '0 9 * * 1-8', - '0 9 * * 1,9', - '0 9 * * MON', - '0 9 *', - '0 9 * * * 2026', - ].map((cron) => cronInWords(cron)), - ).toEqual(Array.from({ length: 13 })); - }); -}); diff --git a/capabilities/coordination/src/overview/trigger-words.ts b/capabilities/coordination/src/overview/trigger-words.ts deleted file mode 100644 index 7968236b2..000000000 --- a/capabilities/coordination/src/overview/trigger-words.ts +++ /dev/null @@ -1,59 +0,0 @@ -import { - durationInWords, - eventFilterPhrases, - listedPhrases, - type Phrases, - type Trigger, - type TriggerFilter, -} from '@on.auto/definitions'; -import { alternatives, capitalized } from '@on.auto/operations'; - -import { runDefinitionFunction } from '../document/workflow-functions.ts'; -import { cronInWords } from './cron-words.ts'; - -export interface Start { - readonly label: string; - readonly words: Phrases; -} - -const byACall: Start = { - label: 'By a call', - words: ['By a call of ', { code: runDefinitionFunction }, ', from a caller or from a step of another workflow'], -}; - -function filterPhrases({ type, attributes }: TriggerFilter): Phrases { - return ['each time ', ...eventFilterPhrases({ ...attributes, type })]; -} - -function eventStart(filters: readonly TriggerFilter[]): Start { - const each = filters.map((filter) => filterPhrases(filter)); - return { - label: `On ${alternatives(filters.map(({ type }) => type))}`, - words: ['On an event, ', ...listedPhrases(each, 'or')], - }; -} - -function cronStart(expression: string): Start { - const plain = cronInWords(expression); - return plain === undefined - ? { - label: `On the schedule ${expression}`, - words: ['On the cron schedule ', { code: expression }, ', read in UTC'], - } - : { label: capitalized(plain), words: [`On a schedule, ${plain}`] }; -} - -function startOf(trigger: Trigger): Start { - if (trigger.kind === 'event') { - return eventStart(trigger.filters); - } - if (trigger.kind === 'cron') { - return cronStart(trigger.expression); - } - const period = durationInWords(trigger.milliseconds); - return { label: `Every ${period}`, words: [`Every ${period}, counted from when the trigger was saved`] }; -} - -export function startsOf(triggers: readonly Trigger[]): readonly Start[] { - return [...triggers.map((trigger) => startOf(trigger)), byACall]; -} diff --git a/capabilities/coordination/src/overview/workflow-overview.test.ts b/capabilities/coordination/src/overview/workflow-overview.test.ts deleted file mode 100644 index a6344aef3..000000000 --- a/capabilities/coordination/src/overview/workflow-overview.test.ts +++ /dev/null @@ -1,161 +0,0 @@ -import { readFileSync } from 'node:fs'; - -import { markdownOf } from '@on.auto/definitions'; -import { overviewAsRead, sectionIn, type HeldDefinition } from '@on.auto/definitions/testing'; -import { Effect } from 'effect'; -import { describe, expect, it } from 'vitest'; - -import { summaryOf } from '../document/workflow-summary.ts'; -import { recordingStarts, recordingWorkflows } from '../testing/recording-starts.ts'; -import { workflow } from '../testing/workflows.ts'; -import { workflowOverview } from './workflow-overview.ts'; - -const workflows = recordingWorkflows(recordingStarts().recording); - -const header = "document:\n dsl: '1.0.3'\n namespace: acme\n name: flow\n version: '1.0.0'\n"; - -function blocksOf(page: string, language: string): readonly string[] { - const text = readFileSync(new URL(`../../../../docs/reference/${page}`, import.meta.url), 'utf8'); - return [...text.matchAll(new RegExp(`\`\`\`${language}\\n([\\s\\S]*?)\`\`\``, 'gu'))].map( - ([, body = '']: readonly string[]) => body, - ); -} - -function descriptionIn(page: string, block: number): string { - return ( - /^description: (?.*)$/mu.exec(blocksOf(page, 'markdown')[block] ?? '')?.groups?.['description'] ?? '' - ); -} - -const heldByTheReferences: readonly HeldDefinition[] = [ - { type: 'reasoning', name: 'review-campaign-brief', description: descriptionIn('workflow-format.md', 0) }, - { type: 'interaction', name: 'approve-campaign-brief', description: descriptionIn('workflow-format.md', 1) }, - { type: 'prediction', name: 'weekly-orders', description: descriptionIn('prediction-format.md', 0) }, - { type: 'computation', name: 'campaign-pace', description: descriptionIn('computation-format.md', 0) }, -]; - -function goldenOf(name: string): string { - return readFileSync(new URL(name, import.meta.url), 'utf8'); -} - -function overviewOf(source: string, name = 'flow'): string { - const { summary, overview } = Effect.runSync(workflows.prepare(source)); - const details = summary.details === undefined ? {} : { details: summary.details }; - return overviewAsRead({ type: 'workflow', name, overview: overview(name), ...details }, heldByTheReferences); -} - -const [reviewAndApprove = '', schedule = ''] = blocksOf('workflow-format.md', 'yaml'); - -const [, nextWeekOrders = ''] = blocksOf('prediction-format.md', 'yaml'); - -const [, campaignPaceTable = ''] = blocksOf('output-shapes.md', 'yaml'); - -describe('the overview of the examples of the references of workflows, prediction functions and output shapes', () => { - it('is, byte for byte, the overview pinned beside this test for each, as read beside the functions the references save', () => { - expect(overviewOf(reviewAndApprove, 'review-and-approve')).toBe(goldenOf('reference-example.overview.md')); - expect(overviewOf(nextWeekOrders, 'next-week-orders')).toBe(goldenOf('prediction-example.overview.md')); - expect(overviewOf(campaignPaceTable, 'campaign-pace-table')).toBe(goldenOf('shaped-example.overview.md')); - }); -}); - -describe('the overview of a workflow whose output names a shape', () => { - it('says the shape in what it produces, in place of the fields of its schema', () => { - const overview = overviewOf(`${header}output: { shape: text }\ndo:\n - write: { set: { text: done } }\n`); - - expect(sectionIn(overview, 'What it produces')).toBe('It answers a **text**: Markdown of at most 65,536 bytes.'); - }); -}); - -describe('the overview of a workflow with a schedule', () => { - const overview = overviewOf(`${header}${schedule}do:\n - greet: { set: { greeting: hello } }\n`); - - it('says each trigger of the schedule of the reference in words, and that a call starts it too', () => { - expect(sectionIn(overview, 'What starts it')).toBe( - [ - '- On an event, each time an event `com.example.brief.submitted` is published to the playbook, when `$data.priority === "high"` holds for its data', - '- On a schedule, at 09:00 UTC from Monday to Friday', - '- Every 15 minutes, counted from when the trigger was saved', - '- By a call of `run_definition`, from a caller or from a step of another workflow', - ].join('\n'), - ); - }); - - it('enters its first step from each trigger, and says nothing of what it calls, since it calls nothing', () => { - expect(sectionIn(overview, 'What it does').split('\n').slice(2, 11)).toEqual([ - ' n1(["On com.example.brief.submitted"])', - ' n2(["At 09:00 UTC from Monday to Friday"])', - ' n3(["Every 15 minutes"])', - ' n4(["By a call"])', - ' n5["1. Set greeting"]', - ' n1 --> n5', - ' n2 --> n5', - ' n3 --> n5', - ' n4 --> n5', - ]); - expect(overview).not.toContain('## What it calls'); - }); -}); - -describe('the overview of a workflow that calls definitions', () => { - const source = [ - `${header}output:`, - ' schema:', - ' document: { type: object, properties: { review: { type: string } }, required: [review] }', - 'do:', - ' - first: { call: run_definition, with: { type: reasoning, name: review-brief } }', - ' - again: { call: run_definition, with: { type: reasoning, name: review-brief } }', - " - chosen: { call: run_definition, with: { type: '${ $data.type }', name: review-brief } }", - ' - pace: { call: run_definition, with: { type: computation, name: campaign-pace } }', - ].join('\n'); - - it('says, as read, each definition its steps run once, with its type, and leaves a definition named when a step runs to that step', () => { - const overview = overviewOf(source); - - expect(sectionIn(overview, 'What it produces')).toBe( - '| Field | Required | What it is |\n| --- | --- | --- |\n| **review** | yes | Text. |', - ); - expect(sectionIn(overview, 'What it calls')).toBe( - [ - 'It calls the reasoning function **Review brief**, which the playbook does not have yet.', - 'It calls the computation function **Campaign pace**: Spend, pace and projection per campaign, in cents, for a reporting period.', - ].join('\n\n'), - ); - expect(overview.split('\n\n## Details\n\n')[1]).toBe( - '### What it calls\n\n- `review-brief`, a reasoning function\n- `campaign-pace`, a computation function\n', - ); - expect(sectionIn(overview, 'What it does')).toContain( - '3. **chosen** runs the definition its call names when it runs.', - ); - }); - - it('composes nothing of what it calls itself, since the read says it', () => { - expect(markdownOf(Effect.runSync(workflows.prepare(source)).overview('flow'))).not.toContain('What it calls'); - }); -}); - -describe('the overview of a workflow without steps', () => { - it('says so in one line', () => { - const document = workflow('do: []'); - const parts = workflowOverview({ document, locate: () => ({ line: 1, column: 1 }) }, 'idle', { - summary: summaryOf(document), - }); - const written = markdownOf({ title: 'idle', opening: [], ...parts }); - - expect(written.endsWith('## What it does\n\nIt has no steps, so a run ends as it starts.\n')).toBe(true); - }); -}); - -describe('the overview of a workflow whose document has an about', () => { - it('writes the about as its author wrote it, after the summary and before what starts it', () => { - const about = [' about: |', ' It serves the front desk.', '', ' It runs on each arrival.']; - const source = `${header} summary: Greets a guest\n metadata:\n${about.join('\n')}\ndo:\n - greet: { set: { a: 1 } }\n`; - - expect(overviewOf(source).split('\n\n').slice(0, 5)).toEqual([ - '# flow', - 'Greets a guest', - 'It serves the front desk.', - 'It runs on each arrival.', - '## What starts it', - ]); - }); -}); diff --git a/capabilities/coordination/src/overview/workflow-overview.ts b/capabilities/coordination/src/overview/workflow-overview.ts deleted file mode 100644 index 6cca4aaa7..000000000 --- a/capabilities/coordination/src/overview/workflow-overview.ts +++ /dev/null @@ -1,46 +0,0 @@ -import { - bulletList, - inputSection, - numberedList, - outputSection, - paragraph, - sectionOf, - type OverviewContext, - type OverviewParts, - type OverviewSection, -} from '@on.auto/definitions'; -import { field, type JsonObject } from '@on.auto/workflow-engine'; - -import type { ReadWorkflow } from '../document/workflow-document.ts'; -import { outputShapeOf } from '../document/workflow-output.ts'; -import { taskDiagramOf } from '../steps/task-diagram.ts'; -import { startsOf } from './trigger-words.ts'; - -function doneSection(document: JsonObject, entries: readonly string[]): OverviewSection { - const { diagram, steps } = taskDiagramOf(field(document, 'do'), entries); - return steps.length === 0 - ? sectionOf('What it does', paragraph('It has no steps, so a run ends as it starts.')) - : sectionOf('What it does', diagram, paragraph('Its steps, numbered as in the diagram:'), numberedList(steps)); -} - -export function workflowOverview( - { document }: ReadWorkflow, - _name: string, - { summary }: OverviewContext, -): OverviewParts { - const { inputSchema, outputSchema, triggers = [] } = summary; - const starts = startsOf(triggers); - return { - body: [], - sections: [ - sectionOf('What starts it', bulletList(starts.map(({ words }) => words))), - inputSection(inputSchema), - outputSection(outputSchema, outputShapeOf(document)), - doneSection( - document, - starts.map(({ label }) => label), - ), - ], - appendix: [], - }; -} diff --git a/capabilities/coordination/src/steps/flow-words.ts b/capabilities/coordination/src/steps/flow-words.ts deleted file mode 100644 index 13127ebde..000000000 --- a/capabilities/coordination/src/steps/flow-words.ts +++ /dev/null @@ -1,105 +0,0 @@ -import { listedPhrases, named, type Phrases } from '@on.auto/definitions'; -import { listed } from '@on.auto/operations'; -import { entriesOf, field, textField, type JsonObject } from '@on.auto/workflow-engine'; - -import type { PlacedTask } from './placed-tasks.ts'; -import { casesOf, type SwitchCase } from './task-flow.ts'; -import { expressionText, objectAt } from './task-reading.ts'; - -export interface StepPlace { - readonly siblings: readonly PlacedTask[]; - readonly parent?: PlacedTask; -} - -function stepsWords(first: number, last: number): string { - if (last < first) { - return 'nothing'; - } - return first === last ? `step ${first}` : `steps ${first} to ${last}`; -} - -export function movedWords(next: string | undefined, { siblings, parent }: StepPlace): string | undefined { - if (next === 'end') { - return 'ends the run'; - } - if (next === 'exit') { - return parent === undefined ? 'ends the run' : `leaves step ${parent.step}`; - } - const target = siblings.find(({ name }) => name === next); - return target === undefined ? undefined : `goes to step ${target.step}`; -} - -export function forWords({ task, step, last }: PlacedTask): Phrases { - const loop = objectAt(task, 'for'); - const condition = field(task, 'while'); - return [ - `does ${stepsWords(step + 1, last)} for each `, - named(textField(loop, 'each') ?? 'item'), - ' of ', - { code: expressionText(field(loop, 'in') ?? null) }, - ...(condition === undefined ? [] : [', while ', { code: expressionText(condition) }, ' holds']), - ]; -} - -export function doWords({ step, last }: PlacedTask): Phrases { - return [`does ${stepsWords(step + 1, last)} in order`]; -} - -export function forkWords({ task, body }: PlacedTask): Phrases { - if (body.length === 0) { - return ['does nothing at once']; - } - const branches = listed(body.map(({ step }) => String(step))); - const handed = - field(objectAt(task, 'fork'), 'compete') === true - ? 'the output of the first to finish, cancelling the others' - : 'a list of their outputs'; - return [`does ${body.length === 1 ? 'step' : 'steps'} ${branches} at once, and hands on ${handed}`]; -} - -function caughtWords(caught: JsonObject): Phrases { - const tests = entriesOf(objectAt(caught, 'errors', 'with')).map(([key, value]): Phrases => - key === 'status' ? [`with status ${expressionText(value)}`] : [`whose ${key} is `, { code: expressionText(value) }], - ); - const when = field(caught, 'when'); - const exceptWhen = field(caught, 'exceptWhen'); - return [ - ...(tests.length === 0 ? ['any error'] : ['an error ', ...listedPhrases(tests, 'and')]), - ...(when === undefined ? [] : [', when ', { code: expressionText(when) }]), - ...(exceptWhen === undefined ? [] : [', except when ', { code: expressionText(exceptWhen) }]), - ]; -} - -function retryWords(caught: JsonObject): Phrases { - const retry = field(caught, 'retry'); - if (retry === undefined) { - return []; - } - const policy: Phrases = typeof retry === 'string' ? ['under the retry policy ', named(retry)] : ['as its catch says']; - return ['tries again ', ...policy, ', and once no retry is left, it ']; -} - -export function tryWords({ task, step, bodyLast, last, recovery }: PlacedTask): Phrases { - const caught = objectAt(task, 'catch'); - const recovered = recovery.length === 0 ? 'hands on its input' : `does ${stepsWords(bodyLast + 1, last)}`; - return [ - `tries ${stepsWords(step + 1, bodyLast)}; on `, - ...caughtWords(caught), - ', it ', - ...retryWords(caught), - recovered, - ]; -} - -function caseWords({ name, when, next }: SwitchCase, place: StepPlace): Phrases { - const chosen: Phrases = when === undefined ? [', by default,'] : [', when ', { code: when }, ',']; - return [named(name), ...chosen, ` ${movedWords(next, place) ?? 'goes on'}`]; -} - -export function switchWords({ task }: PlacedTask, place: StepPlace): Phrases { - const cases = casesOf(task).map((each) => caseWords(each, place)); - const unmatched: readonly Phrases[] = casesOf(task).every(({ when }) => when !== undefined) - ? [[`when no case holds, it ${movedWords(textField(task, 'then'), place) ?? 'goes on'}`]] - : []; - return ['chooses the first case that holds: ', ...listedPhrases([...cases, ...unmatched], 'and')]; -} diff --git a/capabilities/coordination/src/steps/grouped-tasks.test.ts b/capabilities/coordination/src/steps/grouped-tasks.test.ts deleted file mode 100644 index 1cc8bb3fb..000000000 --- a/capabilities/coordination/src/steps/grouped-tasks.test.ts +++ /dev/null @@ -1,155 +0,0 @@ -import { describe, expect, it } from 'vitest'; - -import { doneIn, stepNumbersOf, type DoneSteps } from '../testing/done-steps.ts'; - -interface Example extends DoneSteps { - readonly tasks: readonly string[]; -} - -function drawnFrom({ tasks }: Example): DoneSteps { - return doneIn(tasks.join('\n')); -} - -function expectedOf({ diagram, steps }: Example): DoneSteps { - return { diagram, steps }; -} - -const choice: Example = { - tasks: [ - ' - route:', - ' switch:', - " - large: { when: '${ $data.n > 5 }', then: big }", - ' - fallback: { then: small }', - ' - big: { set: { size: large } }', - ' - small: { set: { size: small } }', - ], - diagram: [ - ' n1(["By a call"])', - ' n2{"1. Choose a case"}', - ' n3["2. Set size"]', - ' n4["3. Set size"]', - ' n1 --> n2', - ' n2 -->|"large"| n3', - ' n2 -->|"fallback"| n4', - ' n3 --> n4', - ], - steps: [ - '1. **route** chooses the first case that holds: **large**, when `$data.n > 5`, goes to step 2 and **fallback**, by default, goes to step 3.', - '2. **big** sets **size**.', - '3. **small** sets **size**.', - ], -}; - -const loop: Example = { - tasks: [ - ' - total:', - " for: { each: line, in: '${ $data.lines }' }", - " while: '${ ($data.sum ?? 0) < 1000 }'", - ' do:', - " - add: { set: { sum: '${ ($data.sum ?? 0) + $line.amount }' } }", - ' - report: { set: { reported: true } }', - ], - diagram: [ - ' subgraph g1["1. For each line of $data.lines"]', - ' n2["2. Set sum"]', - ' end', - ' n1(["By a call"])', - ' n3["3. Set reported"]', - ' n1 --> n2', - ' n2 --> n3', - ' n2 -->|"next item"| n2', - ], - steps: [ - '1. **total** does step 2 for each **line** of `$data.lines`, while `($data.sum ?? 0) < 1000` holds.', - '2. **add** sets **sum**.', - '3. **report** sets **reported**.', - ], -}; - -const branches: Example = { - tasks: [ - ' - reviews:', - ' fork:', - ' branches:', - ' - first: { call: run_definition, with: { type: reasoning, name: review-brief } }', - ' - second: { call: run_definition, with: { type: computation, name: campaign-pace } }', - ' - done: { set: { reviewed: true } }', - ], - diagram: [ - ' subgraph g1["1. At once"]', - ' n2["2. Run review-brief"]', - ' n3["3. Run campaign-pace"]', - ' end', - ' n1(["By a call"])', - ' n4["4. Set reviewed"]', - ' n1 --> n2', - ' n1 --> n3', - ' n2 --> n4', - ' n3 --> n4', - ], - steps: [ - '1. **reviews** does steps 2 and 3 at once, and hands on a list of their outputs.', - '2. **first** runs the reasoning function **review-brief**.', - '3. **second** runs the computation function **campaign-pace**.', - '4. **done** sets **reviewed**.', - ], -}; - -const race: Example = { - tasks: [ - ' - race:', - ' fork:', - ' compete: true', - ' branches:', - ' - quick: { wait: PT1M }', - ' - slow: { wait: { hours: 2 } }', - ], - diagram: [ - ' subgraph g1["1. At once, the first to finish wins"]', - ' n2["2. Wait 1 minute"]', - ' n3["3. Wait 2 hours"]', - ' end', - ' n1(["By a call"])', - ' n1 --> n2', - ' n1 --> n3', - ], - steps: [ - '1. **race** does steps 2 and 3 at once, and hands on the output of the first to finish, cancelling the others.', - '2. **quick** waits 1 minute.', - '3. **slow** waits 2 hours.', - ], -}; - -describe('the diagram of a switch', () => { - it('is a decision with a branch for each case, to the step each case names', () => { - const done = drawnFrom(choice); - - expect(done).toEqual(expectedOf(choice)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); - -describe('the diagram of a for', () => { - it('is a group around its tasks, the last of which goes back for the next item', () => { - const done = drawnFrom(loop); - - expect(done).toEqual(expectedOf(loop)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); - -describe('the diagram of a fork', () => { - it('is a group with a branch for each of its branches, which join at the step after it', () => { - const done = drawnFrom(branches); - - expect(done).toEqual(expectedOf(branches)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); - - it('notes in its group when the first branch to finish wins', () => { - const done = drawnFrom(race); - - expect(done).toEqual(expectedOf(race)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); diff --git a/capabilities/coordination/src/steps/incomplete-tasks.test.ts b/capabilities/coordination/src/steps/incomplete-tasks.test.ts deleted file mode 100644 index c922e6543..000000000 --- a/capabilities/coordination/src/steps/incomplete-tasks.test.ts +++ /dev/null @@ -1,116 +0,0 @@ -import { describe, expect, it } from 'vitest'; - -import { doneIn, stepNumbersOf } from '../testing/done-steps.ts'; - -const tasks = [ - ' - odd: { mystery: 1 }', - ' - loop: { for: {} }', - ' - empty: { fork: { branches: [] } }', - ' - bare: { fork: [] }', - ' - deaf: { listen: {} }', - ' - silent: { raise: {} }', - ' - unnamed: { raise: { error: unavailable } }', - ' - untitled: { raise: { error: { type: https://example.com/errors/x } } }', - ' - quiet: { raise: { error: { status: 500 } } }', - ' - blank: { emit: { event: { with: {} } } }', - " - computed: { set: '${ $data }' }", - ' - nothing: { set: [] }', - ' - fetch: { call: http, with: {} }', - ' - process: { run: {} }', - ' - unguarded: { try: [] }', - ' - none: { do: [] }', - ' - single: { fork: { branches: [{ only: { set: { o: 1 } } }] } }', - ' - bad: { switch: {} }', - ' - mixed: { switch: [1, { fallback: null }] }', - " - maybe: { switch: [{ perhaps: { when: '${ $data.maybe }' } }] }", - ' - idle: { do: [], then: idle }', -]; - -const groups = [ - ' subgraph g1["2. For each item of null"]', - ' end', - ' subgraph g2["3. At once"]', - ' end', - ' subgraph g3["4. At once"]', - ' end', - ' subgraph g4["15. Try"]', - ' end', - ' subgraph g5["16. In order"]', - ' end', - ' subgraph g6["17. At once"]', - ' n13["18. Set o"]', - ' end', - ' subgraph g7["22. In order"]', - ' end', -]; - -const nodes = [ - ' n1(["By a call"])', - ' n2["1. odd"]', - ' n3["5. Wait for an event sent to the run"]', - ' n4(["6. Raise an error"])', - ' n5(["7. Raise unavailable"])', - ' n6(["8. Raise an error"])', - ' n7(["9. Raise an error"])', - ' n8["10. Publish an event"]', - ' n9["11. Set its data"]', - ' n10["12. Set its data"]', - ' n11["13. Call http"]', - ' n12["14. Run a process"]', - ' n14{"19. Choose a case"}', - ' n15{"20. Choose a case"}', - ' n16{"21. Choose a case"}', -]; - -const edges = [ - ' n1 --> n2', - ' n2 --> n3', - ' n3 --> n4', - ' n8 --> n9', - ' n9 --> n10', - ' n10 --> n11', - ' n11 --> n12', - ' n12 --> n13', - ' n13 --> n14', - ' n14 -->|"no case holds"| n15', - ' n15 -->|"fallback"| n16', -]; - -const steps = [ - '1. **odd** is a step this runtime does not run.', - '2. **loop** does nothing for each **item** of `null`.', - '3. **empty** does nothing at once.', - '4. **bare** does nothing at once.', - '5. **deaf** waits for an event sent to the run.', - '6. **silent** raises an error.', - '7. **unnamed** raises the error **unavailable** its document names under `use.errors`.', - '8. **untitled** raises an error of the type `https://example.com/errors/x`.', - '9. **quiet** raises an error.', - '10. **blank** publishes an event of a type computed when it runs to the playbook.', - '11. **computed** sets its data from `$data`.', - '12. **nothing** sets its data.', - '13. **fetch** calls `http`.', - '14. **process** runs a process.', - '15. **unguarded** tries nothing; on any error, it hands on its input.', - '16. **none** does nothing in order.', - '17. **single** does step 18 at once, and hands on a list of their outputs.', - '18. **only** sets **o**.', - '19. **bad** chooses the first case that holds: when no case holds, it goes on.', - '20. **mixed** chooses the first case that holds: **fallback**, by default, goes on.', - '21. **maybe** chooses the first case that holds: **perhaps**, when `$data.maybe`, goes on and when no case holds, it goes on.', - '22. **idle** does nothing in order, and then goes to step 22.', -]; - -describe('the diagram of steps its document leaves incomplete', () => { - it('still draws each step, says what it can of it, and draws no edge around a step that comes back to itself', () => { - const done = doneIn(tasks.join('\n')); - const { drawn, listed } = stepNumbersOf(done); - - expect(done).toEqual({ diagram: [...groups, ...nodes, ...edges], steps }); - expect(drawn).toEqual(listed); - }); - - it('has no steps when its list holds none', () => { - expect(doneIn(' []')).toEqual({ diagram: [' n1(["By a call"])'], steps: [''] }); - }); -}); diff --git a/capabilities/coordination/src/steps/leaf-tasks.test.ts b/capabilities/coordination/src/steps/leaf-tasks.test.ts deleted file mode 100644 index 2a0be8132..000000000 --- a/capabilities/coordination/src/steps/leaf-tasks.test.ts +++ /dev/null @@ -1,178 +0,0 @@ -import { describe, expect, it } from 'vitest'; - -import { doneIn, stepNumbersOf, type DoneSteps } from '../testing/done-steps.ts'; - -interface Example extends DoneSteps { - readonly tasks: readonly string[]; - readonly entries?: readonly string[]; -} - -function drawnFrom({ tasks, entries }: Example): DoneSteps { - return doneIn(tasks.join('\n'), entries); -} - -function expectedOf({ diagram, steps }: Example): DoneSteps { - return { diagram, steps }; -} - -const called: Example = { - tasks: [ - ' - review:', - ' call: run_definition', - " with: { type: reasoning, name: review-brief, input: { brief: '${ $data.brief }' } }", - ], - entries: ['On com.example.brief.submitted', 'By a call'], - diagram: [ - ' n1(["On com.example.brief.submitted"])', - ' n2(["By a call"])', - ' n3["1. Run review-brief"]', - ' n1 --> n3', - ' n2 --> n3', - ], - steps: ['1. **review** runs the reasoning function **review-brief**.'], -}; - -const assigned: Example = { - tasks: [" - tally: { set: { count: 1, total: '${ $data.total }' } }"], - diagram: [' n1(["By a call"])', ' n2["1. Set count and total"]', ' n1 --> n2'], - steps: ['1. **tally** sets **count** and **total**.'], -}; - -const raised: Example = { - tasks: [ - ' - stop:', - ' raise:', - ' error: { type: https://example.com/errors/stopped, status: 409, title: Stopped }', - ' - unreached: { set: { reached: true } }', - ], - diagram: [' n1(["By a call"])', ' n2(["1. Raise “Stopped”"])', ' n3["2. Set reached"]', ' n1 --> n2'], - steps: ['1. **stop** raises the error *Stopped*, with status 409.', '2. **unreached** sets **reached**.'], -}; - -const waited: Example = { - tasks: [ - ' - hear:', - ' listen:', - ' to:', - ' any:', - ' - with: { type: com.example.a }', - ` - with: { type: '\${ "com.example." + $data.kind }' }`, - ' timeout: { after: PT1H }', - ' - pause: { wait: { hours: 2, minutes: 30 } }', - ' - announce:', - ' emit:', - ' event:', - ' with: { type: com.example.brief.approved, source: https://example.com/campaigns }', - ], - diagram: [ - ' n1(["By a call"])', - ' n2["1. Wait for com.example.a or an event sent to the run"]', - ' n3["2. Wait 2 hours and 30 minutes"]', - ' n4["3. Publish com.example.brief.approved"]', - ' n1 --> n2', - ' n2 --> n3', - ' n3 --> n4', - ], - steps: [ - '1. **hear** waits for the first of the event **com.example.a** or an event sent to the run, for at most 1 hour.', - '2. **pause** waits 2 hours and 30 minutes.', - '3. **announce** publishes the event **com.example.brief.approved** to the playbook.', - ], -}; - -const heard: Example = { - tasks: [ - ' - one: { listen: { to: { one: { with: { type: com.example.a } } } } }', - ' - all: { listen: { to: { all: [{ with: { type: com.example.a } }, { with: { type: com.example.b } }] } } }', - ], - diagram: [ - ' n1(["By a call"])', - ' n2["1. Wait for com.example.a"]', - ' n3["2. Wait for com.example.a and com.example.b"]', - ' n1 --> n2', - ' n2 --> n3', - ], - steps: [ - '1. **one** waits for the event **com.example.a**.', - '2. **all** waits for each of the event **com.example.a** and the event **com.example.b**.', - ], -}; - -const ordered: Example = { - tasks: [ - ' - steps:', - ' do:', - ' - first: { set: { a: 1 } }', - ' - second: { set: { b: 2 } }', - " - build: { run: { script: { language: js, code: '1' } } }", - ], - diagram: [ - ' subgraph g1["1. In order"]', - ' n2["2. Set a"]', - ' n3["3. Set b"]', - ' end', - ' n1(["By a call"])', - ' n4["4. Run a script"]', - ' n1 --> n2', - ' n2 --> n3', - ' n3 --> n4', - ], - steps: [ - '1. **steps** does steps 2 to 3 in order.', - '2. **first** sets **a**.', - '3. **second** sets **b**.', - '4. **build** runs a script.', - ], -}; - -describe('the diagram of a call of run_definition', () => { - it('is a node named for the definition it runs, entered from each way the workflow starts', () => { - const done = drawnFrom(called); - - expect(done).toEqual(expectedOf(called)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); - -describe('the diagram of a set', () => { - it('is a node that names what it sets', () => { - const done = drawnFrom(assigned); - - expect(done).toEqual(expectedOf(assigned)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); - -describe('the diagram of a raise', () => { - it('is an end node, with no step after it', () => { - const done = drawnFrom(raised); - - expect(done).toEqual(expectedOf(raised)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); - -describe('the diagram of a listen, a wait and an emit', () => { - it('are nodes that wait for their events, wait their time and publish their event', () => { - const done = drawnFrom(waited); - - expect(done).toEqual(expectedOf(waited)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); - - it('say which events a listen waits for, one or each of them', () => { - const done = drawnFrom(heard); - - expect(done).toEqual(expectedOf(heard)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); - -describe('the diagram of a do and of a run', () => { - it('are a group of the tasks in order, and a node that runs its process', () => { - const done = drawnFrom(ordered); - - expect(done).toEqual(expectedOf(ordered)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); diff --git a/capabilities/coordination/src/steps/placed-tasks.ts b/capabilities/coordination/src/steps/placed-tasks.ts deleted file mode 100644 index e41f13a9f..000000000 --- a/capabilities/coordination/src/steps/placed-tasks.ts +++ /dev/null @@ -1,123 +0,0 @@ -import type { DiagramGroup, DiagramNode, NodeShape } from '@on.auto/definitions'; -import { - field, - kindOf, - taskEntries, - type Json, - type JsonObject, - type TaskEntry, - type TaskKind, -} from '@on.auto/workflow-engine'; - -import { objectAt } from './task-reading.ts'; - -export type LabelOf = (kind: TaskKind | undefined, task: JsonObject, name: string) => string; - -export interface NumberedTask { - readonly name: string; - readonly task: JsonObject; - readonly kind: TaskKind | undefined; - readonly step: number; - readonly body: readonly NumberedTask[]; - readonly recovery: readonly NumberedTask[]; - readonly bodyLast: number; - readonly last: number; -} - -export interface PlacedTask extends NumberedTask { - readonly node?: number; - readonly group?: number; - readonly body: readonly PlacedTask[]; - readonly recovery: readonly PlacedTask[]; -} - -export interface Placement { - readonly tasks: readonly PlacedTask[]; - readonly nodes: readonly DiagramNode[]; - readonly groups: readonly DiagramGroup[]; -} - -interface GroupDraft { - readonly label: string; - readonly within: number | undefined; -} - -type GroupedKind = 'for' | 'do' | 'try' | 'fork'; - -const groupedKinds: ReadonlySet = new Set(['for', 'do', 'try', 'fork']); - -const shapeOfKind: Partial>> = { switch: 'decision', raise: 'terminal' }; - -function bodyOf(kind: TaskKind | undefined, task: JsonObject): Json | undefined { - if (kind === 'fork') { - return field(objectAt(task, 'fork'), 'branches'); - } - return field(task, kind === 'try' ? 'try' : 'do'); -} - -function recoveryOf(kind: TaskKind | undefined, task: JsonObject): Json | undefined { - return kind === 'try' ? field(objectAt(task, 'catch'), 'do') : undefined; -} - -function nodeOf(label: string, kind: TaskKind | undefined): DiagramNode { - const shape = kind === undefined ? undefined : shapeOfKind[kind]; - return shape === undefined ? { label } : { label, shape }; -} - -function groupOf({ label, within }: GroupDraft, nodes: readonly number[]): DiagramGroup { - return within === undefined ? { label, nodes } : { label, nodes, within }; -} - -export function numberedTasksOf(list: Json | undefined): readonly NumberedTask[] { - let steps = 0; - - function numberedList(tasks: Json | undefined): readonly NumberedTask[] { - return taskEntries(tasks, '').map((entry) => numberedTask(entry)); - } - - function numberedTask({ name, task }: TaskEntry): NumberedTask { - steps += 1; - const step = steps; - const kind = kindOf(task); - if (!groupedKinds.has(kind)) { - return { name, task, kind, step, body: [], recovery: [], bodyLast: step, last: step }; - } - const body = numberedList(bodyOf(kind, task)); - const bodyLast = steps; - const recovery = numberedList(recoveryOf(kind, task)); - return { name, task, kind, step, body, recovery, bodyLast, last: steps }; - } - - return numberedList(list); -} - -export function placedTasksOf(list: Json | undefined, entries: number, labelOf: LabelOf): Placement { - const nodes: DiagramNode[] = []; - const nodeGroups: (number | undefined)[] = []; - const drafts: GroupDraft[] = []; - - function placedTask(numbered: NumberedTask, within?: number): PlacedTask { - const { name, task, kind, step } = numbered; - const label = `${step}. ${labelOf(kind, task, name)}`; - if (!groupedKinds.has(kind)) { - const node = entries + nodes.length; - nodes.push(nodeOf(label, kind)); - nodeGroups.push(within); - return { ...numbered, node, body: [], recovery: [] }; - } - const group = drafts.length; - drafts.push({ label, within }); - const body = numbered.body.map((inner) => placedTask(inner, group)); - const recovery = numbered.recovery.map((inner) => placedTask(inner, within)); - return { ...numbered, group, body, recovery }; - } - - const tasks = numberedTasksOf(list).map((numbered) => placedTask(numbered)); - const groups = drafts.map((draft, group) => - groupOf( - draft, - nodeGroups.flatMap((within, index) => (within === group ? [entries + index] : [])), - ), - ); - return { tasks, nodes, groups }; -} diff --git a/capabilities/coordination/src/steps/task-diagram.ts b/capabilities/coordination/src/steps/task-diagram.ts deleted file mode 100644 index 1a7f38db5..000000000 --- a/capabilities/coordination/src/steps/task-diagram.ts +++ /dev/null @@ -1,97 +0,0 @@ -import { named, type Diagram, type Phrases } from '@on.auto/definitions'; -import { field, textField, type Json, type JsonObject, type TaskKind } from '@on.auto/workflow-engine'; - -import { doWords, forkWords, forWords, movedWords, switchWords, tryWords, type StepPlace } from './flow-words.ts'; -import { placedTasksOf, type PlacedTask } from './placed-tasks.ts'; -import { edgesOf } from './task-edges.ts'; -import { conditionOf } from './task-flow.ts'; -import { - callLabel, - emitLabel, - forkLabel, - forLabel, - listenLabel, - raiseLabel, - runLabel, - setLabel, - waitLabel, -} from './task-labels.ts'; -import { durationText, objectAt } from './task-reading.ts'; -import { callWords, emitWords, listenWords, raiseWords, runWords, setWords, waitWords } from './task-words.ts'; - -export interface TaskDiagram { - readonly diagram: Diagram; - readonly steps: readonly Phrases[]; -} - -interface KindWords { - readonly label: (task: JsonObject) => string; - readonly words: (task: PlacedTask, place: StepPlace) => Phrases; -} - -const wordsOfKind: Readonly> = { - for: { label: forLabel, words: forWords }, - try: { label: () => 'Try', words: tryWords }, - call: { label: callLabel, words: ({ task }) => callWords(task) }, - fork: { label: forkLabel, words: forkWords }, - emit: { label: emitLabel, words: ({ task }) => emitWords(task) }, - listen: { label: listenLabel, words: ({ task }) => listenWords(task) }, - raise: { label: raiseLabel, words: ({ task }) => raiseWords(task) }, - run: { label: runLabel, words: ({ task }) => runWords(task) }, - set: { label: setLabel, words: ({ task }) => setWords(task) }, - switch: { label: () => 'Choose a case', words: switchWords }, - wait: { label: waitLabel, words: ({ task }) => waitWords(task) }, - do: { label: () => 'In order', words: doWords }, -}; - -const unmoved: ReadonlySet = new Set(['switch', 'raise']); - -function labelOf(kind: TaskKind | undefined, task: JsonObject, name: string): string { - return kind === undefined ? name : wordsOfKind[kind].label(task); -} - -function timeoutWords(task: JsonObject): Phrases { - const timeout = field(task, 'timeout'); - if (timeout === undefined) { - return []; - } - if (typeof timeout === 'string') { - return [', within the timeout ', named(timeout)]; - } - const duration = durationText(field(objectAt(task, 'timeout'), 'after')); - return [duration === undefined ? ', within a time computed when it runs' : `, for at most ${duration}`]; -} - -function suffixWords(placed: PlacedTask, place: StepPlace): Phrases { - const condition = conditionOf(placed.task); - const moved = unmoved.has(placed.kind) ? undefined : movedWords(textField(placed.task, 'then'), place); - return [ - ...(condition === undefined ? [] : [', only when ', { code: condition }, ' holds']), - ...timeoutWords(placed.task), - ...(moved === undefined ? [] : [`, and then ${moved}`]), - ]; -} - -function itemOf(placed: PlacedTask, place: StepPlace): Phrases { - const words = - placed.kind === undefined ? ['is a step this runtime does not run'] : wordsOfKind[placed.kind].words(placed, place); - return [named(placed.name), ' ', ...words, ...suffixWords(placed, place), '.']; -} - -function stepItems(tasks: readonly PlacedTask[], parent?: PlacedTask): readonly Phrases[] { - return tasks.flatMap((task) => [ - itemOf(task, parent === undefined ? { siblings: tasks } : { siblings: tasks, parent }), - ...stepItems(task.body, task), - ...stepItems(task.recovery, task), - ]); -} - -export function taskDiagramOf(list: Json | undefined, entries: readonly string[]): TaskDiagram { - const { tasks, nodes, groups } = placedTasksOf(list, entries.length, labelOf); - const starts = entries.map((label) => ({ label, shape: 'terminal' as const })); - const edges = edgesOf( - tasks, - entries.map((_, from) => ({ from })), - ); - return { diagram: { kind: 'diagram', nodes: [...starts, ...nodes], edges, groups }, steps: stepItems(tasks) }; -} diff --git a/capabilities/coordination/src/steps/task-edges.ts b/capabilities/coordination/src/steps/task-edges.ts deleted file mode 100644 index db6bfe1a5..000000000 --- a/capabilities/coordination/src/steps/task-edges.ts +++ /dev/null @@ -1,174 +0,0 @@ -import type { DiagramEdge } from '@on.auto/definitions'; -import { textField } from '@on.auto/workflow-engine'; - -import type { PlacedTask } from './placed-tasks.ts'; -import { casesOf, catchLabelOf, conditionOf } from './task-flow.ts'; - -export interface Exit { - readonly from: number; - readonly label?: string; - readonly dashed?: boolean; -} - -type Visited = ReadonlySet; - -type Onward = (exits: readonly Exit[], visited: Visited) => void; - -interface Place { - readonly tasks: readonly PlacedTask[]; - readonly onward: Onward; - readonly leave: Onward; - readonly draw: (edge: DiagramEdge) => void; -} - -interface Spot { - readonly place: Place; - readonly position: number; - readonly task: PlacedTask; -} - -const unvisited: Visited = new Set(); - -function runEnds(): void {} - -function labelled(exits: readonly Exit[], label: string): readonly Exit[] { - return exits.map((exit) => ({ ...exit, label: exit.label === undefined ? label : `${exit.label}, ${label}` })); -} - -function nodesWithin(tasks: readonly PlacedTask[]): readonly number[] { - return tasks.flatMap(({ node, body, recovery }) => [ - ...(node === undefined ? [] : [node]), - ...nodesWithin(body), - ...nodesWithin(recovery), - ]); -} - -function drawn({ from, label, dashed = false }: Exit, to: number): DiagramEdge { - return { from, to, ...(label === undefined ? {} : { label }), ...(dashed ? { dashed } : {}) }; -} - -function routed(next: string | undefined, spot: Spot, exits: readonly Exit[], visited: Visited): void { - const { place, position } = spot; - if (next === 'exit') { - place.leave(exits, visited); - } else if (next !== 'end') { - const target = place.tasks.findIndex(({ name }) => name === next); - arrive(place, target === -1 ? position + 1 : target, exits, visited); - } -} - -function completed(spot: Spot, exits: readonly Exit[], visited: Visited): void { - routed(textField(spot.task.task, 'then'), spot, exits, visited); -} - -function finishing(spot: Spot): Onward { - return (exits, visited) => { - completed(spot, exits, visited); - }; -} - -function bodyPlace(spot: Spot): Place { - const done = finishing(spot); - const looped: Onward = (exits, visited) => { - done(exits, visited); - if (!visited.has(spot.task)) { - arrive(bodyPlace(spot), 0, labelled(exits, 'next item'), new Set([...visited, spot.task])); - } - }; - const { draw } = spot.place; - return { tasks: spot.task.body, onward: spot.task.kind === 'for' ? looped : done, leave: done, draw }; -} - -function branchPlaces(spot: Spot): readonly Place[] { - const done = finishing(spot); - const { draw } = spot.place; - return spot.task.body.map((branch) => ({ tasks: [branch], onward: done, leave: done, draw })); -} - -function innerPlaces(spot: Spot): readonly Place[] { - return spot.task.kind === 'fork' ? branchPlaces(spot) : [bodyPlace(spot)]; -} - -function entered(spot: Spot, exits: readonly Exit[], visited: Visited): void { - const { node, kind, body } = spot.task; - if (node !== undefined) { - for (const exit of exits) { - spot.place.draw(drawn(exit, node)); - } - } else if (kind === 'fork' && body.length === 0) { - completed(spot, exits, visited); - } else { - for (const place of innerPlaces(spot)) { - arrive(place, 0, exits, visited); - } - } -} - -function arrive(place: Place, position: number, exits: readonly Exit[], visited: Visited): void { - const task = place.tasks[position]; - if (task === undefined) { - place.onward(exits, visited); - return; - } - if (visited.has(task)) { - return; - } - const condition = conditionOf(task.task); - const entering = condition === undefined ? exits : labelled(exits, `if ${condition}`); - entered({ place, position, task }, entering, new Set([...visited, task])); - if (condition !== undefined) { - arrive(place, position + 1, labelled(exits, 'otherwise'), visited); - } -} - -function wiredSwitch(spot: Spot, from: number): void { - const cases = casesOf(spot.task.task); - for (const { name, next } of cases) { - routed(next, spot, [{ from, label: name }], unvisited); - } - if (cases.every(({ when }) => when !== undefined)) { - completed(spot, [{ from, label: 'no case holds' }], unvisited); - } -} - -function wiredTry(spot: Spot): void { - const label = catchLabelOf(spot.task.task); - const failing = nodesWithin(spot.task.body).map((from) => ({ from, label, dashed: true })); - const { draw } = spot.place; - const recovery: Place = { tasks: spot.task.recovery, onward: finishing(spot), leave: finishing(spot), draw }; - arrive(recovery, 0, failing, unvisited); - wired(recovery); -} - -function wiredTask(spot: Spot): void { - const { node, kind } = spot.task; - if (node === undefined) { - for (const place of innerPlaces(spot)) { - wired(place); - } - if (kind === 'try') { - wiredTry(spot); - } - } else if (kind === 'switch') { - wiredSwitch(spot, node); - } else if (kind !== 'raise') { - completed(spot, [{ from: node }], unvisited); - } -} - -function wired(place: Place): void { - for (const [position, task] of place.tasks.entries()) { - wiredTask({ place, position, task }); - } -} - -export function edgesOf(tasks: readonly PlacedTask[], entries: readonly Exit[]): readonly DiagramEdge[] { - const edges = new Map(); - const draw = (edge: DiagramEdge) => { - edges.set(JSON.stringify(edge), edge); - }; - const top: Place = { tasks, onward: runEnds, leave: runEnds, draw }; - arrive(top, 0, entries, unvisited); - wired(top); - return [...edges.values()]; -} diff --git a/capabilities/coordination/src/steps/task-flow.test.ts b/capabilities/coordination/src/steps/task-flow.test.ts deleted file mode 100644 index 6cc14ad63..000000000 --- a/capabilities/coordination/src/steps/task-flow.test.ts +++ /dev/null @@ -1,143 +0,0 @@ -import { describe, expect, it } from 'vitest'; - -import { doneIn, stepNumbersOf, type DoneSteps } from '../testing/done-steps.ts'; - -interface Example extends DoneSteps { - readonly tasks: readonly string[]; -} - -function drawnFrom({ tasks }: Example): DoneSteps { - return doneIn(tasks.join('\n')); -} - -function expectedOf({ diagram, steps }: Example): DoneSteps { - return { diagram, steps }; -} - -const jumps: Example = { - tasks: [ - " - start: { set: { n: 1 }, if: '${ $data.go }' }", - ' - loop:', - ' do:', - ' - check: { set: { n: 2 }, then: exit }', - ' - never: { set: { n: 3 } }', - ' - again: { set: { n: 4 }, then: start }', - ], - diagram: [ - ' subgraph g1["2. In order"]', - ' n3["3. Set n"]', - ' n4["4. Set n"]', - ' end', - ' n1(["By a call"])', - ' n2["1. Set n"]', - ' n5["5. Set n"]', - ' n1 -->|"if $data.go"| n2', - ' n1 -->|"otherwise"| n3', - ' n2 --> n3', - ' n3 --> n5', - ' n4 --> n5', - ' n5 -->|"if $data.go"| n2', - ' n5 -->|"otherwise"| n3', - ], - steps: [ - '1. **start** sets **n**, only when `$data.go` holds.', - '2. **loop** does steps 3 to 4 in order.', - '3. **check** sets **n**, and then leaves step 2.', - '4. **never** sets **n**.', - '5. **again** sets **n**, and then goes to step 1.', - ], -}; - -const endings: Example = { - tasks: [ - ' - route:', - ' switch:', - " - stop: { when: '${ $data.stop }', then: end }", - " - leave: { when: '${ $data.leave }', then: exit }", - " - onward: { when: '${ $data.onward }', then: continue }", - ' then: last', - ' - skipped: { set: { skipped: true }, then: end }', - ' - last: { set: { last: true }, then: exit }', - ], - diagram: [ - ' n1(["By a call"])', - ' n2{"1. Choose a case"}', - ' n3["2. Set skipped"]', - ' n4["3. Set last"]', - ' n1 --> n2', - ' n2 -->|"onward"| n3', - ' n2 -->|"no case holds"| n4', - ], - steps: [ - '1. **route** chooses the first case that holds: **stop**, when `$data.stop`, ends the run, **leave**, when `$data.leave`, ends the run, **onward**, when `$data.onward`, goes on, and when no case holds, it goes to step 3.', - '2. **skipped** sets **skipped**, and then ends the run.', - '3. **last** sets **last**, and then ends the run.', - ], -}; - -const conditions: Example = { - tasks: [ - ' - steps:', - ' do:', - ' - route:', - ' switch:', - " - out: { when: '${ $data.out }', then: exit }", - ' - other: {}', - " - guarded: { set: { g: 1 }, if: '${ $data.g }' }", - ' - after: { set: { a: 1 } }', - ' - named: { wait: PT1M, timeout: quick }', - " - computed: { wait: '${ $data.wait }', timeout: { after: '${ $data.most }' } }", - ' - held: { wait: PT1S, timeout: {} }', - ], - diagram: [ - ' subgraph g1["1. In order"]', - ' n2{"2. Choose a case"}', - ' n3["3. Set g"]', - ' end', - ' n1(["By a call"])', - ' n4["4. Set a"]', - ' n5["5. Wait 1 minute"]', - ' n6["6. Wait"]', - ' n7["7. Wait 1 second"]', - ' n1 --> n2', - ' n2 -->|"out"| n4', - ' n2 -->|"other, if $data.g"| n3', - ' n2 -->|"other, otherwise"| n4', - ' n3 --> n4', - ' n4 --> n5', - ' n5 --> n6', - ' n6 --> n7', - ], - steps: [ - '1. **steps** does steps 2 to 3 in order.', - '2. **route** chooses the first case that holds: **out**, when `$data.out`, leaves step 1 and **other**, by default, goes on.', - '3. **guarded** sets **g**, only when `$data.g` holds.', - '4. **after** sets **a**.', - '5. **named** waits 1 minute, within the timeout **quick**.', - '6. **computed** waits for a time computed when it runs, within a time computed when it runs.', - '7. **held** waits 1 second, within a time computed when it runs.', - ], -}; - -describe('the flow between the steps of a diagram', () => { - it('follows then to a named step, out of a list with exit, and around a step whose if does not hold', () => { - const done = drawnFrom(jumps); - - expect(done).toEqual(expectedOf(jumps)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); - - it('ends the run at end and at an exit of the workflow, and follows a switch of its own when no case holds', () => { - const done = drawnFrom(endings); - - expect(done).toEqual(expectedOf(endings)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); - - it('labels a case that reaches a step whose if may not hold, and says the timeout of a step', () => { - const done = drawnFrom(conditions); - - expect(done).toEqual(expectedOf(conditions)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); diff --git a/capabilities/coordination/src/steps/task-flow.ts b/capabilities/coordination/src/steps/task-flow.ts deleted file mode 100644 index 28758e630..000000000 --- a/capabilities/coordination/src/steps/task-flow.ts +++ /dev/null @@ -1,38 +0,0 @@ -import { entriesOf, field, isList, isObject, textField, type Json, type JsonObject } from '@on.auto/workflow-engine'; - -import { expressionText, objectAt } from './task-reading.ts'; - -export interface SwitchCase { - readonly name: string; - readonly when?: string; - readonly next?: string; -} - -function caseOf(name: string, value: Json): SwitchCase { - const written = isObject(value) ? value : {}; - const when = field(written, 'when'); - const next = textField(written, 'then'); - return { - name, - ...(when === undefined ? {} : { when: expressionText(when) }), - ...(next === undefined ? {} : { next }), - }; -} - -export function casesOf(task: JsonObject): readonly SwitchCase[] { - const cases = field(task, 'switch'); - return isList(cases) - ? cases.flatMap((item) => (isObject(item) ? entriesOf(item).map(([name, value]) => caseOf(name, value)) : [])) - : []; -} - -export function conditionOf(task: JsonObject): string | undefined { - const condition = field(task, 'if'); - return condition === undefined ? undefined : expressionText(condition); -} - -export function catchLabelOf(task: JsonObject): string { - const status = field(objectAt(task, 'catch', 'errors', 'with'), 'status'); - const retried = field(objectAt(task, 'catch'), 'retry') === undefined ? '' : ', after retries'; - return typeof status === 'number' ? `on error ${status}${retried}` : `on an error${retried}`; -} diff --git a/capabilities/coordination/src/steps/task-labels.ts b/capabilities/coordination/src/steps/task-labels.ts deleted file mode 100644 index ed81aa7ba..000000000 --- a/capabilities/coordination/src/steps/task-labels.ts +++ /dev/null @@ -1,57 +0,0 @@ -import { alternatives, listed, quoted } from '@on.auto/operations'; -import { field, textField, type JsonObject } from '@on.auto/workflow-engine'; - -import { definitionCallOf, literalText } from '../document/workflow-functions.ts'; -import { durationText, expressionText, listenedOf, namedKeys, objectAt, runKindOf } from './task-reading.ts'; - -export function callLabel(task: JsonObject): string { - const call = definitionCallOf(task); - if (call === undefined) { - return `Call ${String(textField(task, 'call'))}`; - } - return call.name === undefined ? 'Run a definition' : `Run ${call.name}`; -} - -export function setLabel(task: JsonObject): string { - const keys = namedKeys(field(task, 'set')); - return keys === undefined || keys.length === 0 ? 'Set its data' : `Set ${listed(keys)}`; -} - -export function listenLabel(task: JsonObject): string { - const { mode, types } = listenedOf(task); - const named = types.map((type) => type ?? 'an event sent to the run'); - return `Wait for ${mode === 'all' ? listed(named) : alternatives(named)}`; -} - -export function waitLabel(task: JsonObject): string { - const duration = durationText(field(task, 'wait')); - return duration === undefined ? 'Wait' : `Wait ${duration}`; -} - -export function emitLabel(task: JsonObject): string { - const type = literalText(field(objectAt(task, 'emit', 'event', 'with'), 'type')); - return type === undefined ? 'Publish an event' : `Publish ${type}`; -} - -export function raiseLabel(task: JsonObject): string { - const error = field(objectAt(task, 'raise'), 'error'); - if (typeof error === 'string') { - return `Raise ${error}`; - } - const title = textField(objectAt(task, 'raise', 'error'), 'title'); - return title === undefined ? 'Raise an error' : `Raise ${quoted(title)}`; -} - -export function runLabel(task: JsonObject): string { - const kind = runKindOf(task); - return kind === undefined ? 'Run a process' : `Run a ${kind}`; -} - -export function forLabel(task: JsonObject): string { - const loop = objectAt(task, 'for'); - return `For each ${textField(loop, 'each') ?? 'item'} of ${expressionText(field(loop, 'in') ?? null)}`; -} - -export function forkLabel(task: JsonObject): string { - return field(objectAt(task, 'fork'), 'compete') === true ? 'At once, the first to finish wins' : 'At once'; -} diff --git a/capabilities/coordination/src/steps/task-reading.ts b/capabilities/coordination/src/steps/task-reading.ts deleted file mode 100644 index 200953e16..000000000 --- a/capabilities/coordination/src/steps/task-reading.ts +++ /dev/null @@ -1,57 +0,0 @@ -import { durationInWords } from '@on.auto/definitions'; -import { - enclosedBody, - field, - isList, - isObject, - objectField, - readDuration, - type Json, - type JsonObject, -} from '@on.auto/workflow-engine'; - -import { literalText } from '../document/workflow-functions.ts'; - -type ListenMode = 'one' | 'any' | 'all'; - -export interface Listened { - readonly mode: ListenMode; - readonly types: readonly (string | undefined)[]; -} - -const listenModes: readonly ListenMode[] = ['one', 'any', 'all']; - -const runKinds: readonly string[] = ['container', 'script', 'shell', 'workflow']; - -export function objectAt(value: JsonObject, ...keys: readonly string[]): JsonObject { - return keys.reduce((current, key) => objectField(current, key) ?? {}, value); -} - -export function expressionText(value: Json): string { - return typeof value === 'string' ? (enclosedBody(value) ?? value).trim() : JSON.stringify(value); -} - -export function durationText(value: Json | undefined): string | undefined { - const reading = value === undefined ? undefined : readDuration(value); - return reading !== undefined && 'milliseconds' in reading ? durationInWords(reading.milliseconds) : undefined; -} - -function filterType(filter: Json): string | undefined { - return literalText(field(objectAt({ filter }, 'filter', 'with'), 'type')); -} - -export function listenedOf(task: JsonObject): Listened { - const to = objectAt(task, 'listen', 'to'); - const mode = listenModes.find((each) => field(to, each) !== undefined) ?? 'one'; - const filters = field(to, mode); - return { mode, types: isList(filters) ? filters.map((filter) => filterType(filter)) : [filterType(filters ?? {})] }; -} - -export function namedKeys(value: Json | undefined): readonly string[] | undefined { - return isObject(value) ? Object.keys(value) : undefined; -} - -export function runKindOf(task: JsonObject): string | undefined { - const run = objectAt(task, 'run'); - return runKinds.find((kind) => field(run, kind) !== undefined); -} diff --git a/capabilities/coordination/src/steps/task-words.ts b/capabilities/coordination/src/steps/task-words.ts deleted file mode 100644 index ec0ee1cce..000000000 --- a/capabilities/coordination/src/steps/task-words.ts +++ /dev/null @@ -1,79 +0,0 @@ -import { definitionResourceLabel, listedPhrases, named, type Phrases } from '@on.auto/definitions'; -import { field, textField, type JsonObject } from '@on.auto/workflow-engine'; - -import { definitionCallOf, literalText, namesItsDefinition } from '../document/workflow-functions.ts'; -import { durationText, expressionText, listenedOf, namedKeys, objectAt, runKindOf } from './task-reading.ts'; - -export function callWords(task: JsonObject): Phrases { - const call = definitionCallOf(task); - if (call === undefined) { - return ['calls ', { code: String(textField(task, 'call')) }]; - } - return namesItsDefinition(call) - ? [`runs the ${definitionResourceLabel(call.type)} `, named(call.name)] - : ['runs the definition its call names when it runs']; -} - -export function setWords(task: JsonObject): Phrases { - const value = field(task, 'set'); - const keys = namedKeys(value) ?? []; - if (keys.length > 0) { - return [ - 'sets ', - ...listedPhrases( - keys.map((key) => [named(key)]), - 'and', - ), - ]; - } - return typeof value === 'string' ? ['sets its data from ', { code: expressionText(value) }] : ['sets its data']; -} - -export function listenWords(task: JsonObject): Phrases { - const { mode, types } = listenedOf(task); - const events = types.map((type): Phrases => - type === undefined ? ['an event sent to the run'] : ['the event ', named(type)], - ); - if (mode === 'any') { - return ['waits for the first of ', ...listedPhrases(events, 'or')]; - } - return [mode === 'all' ? 'waits for each of ' : 'waits for ', ...listedPhrases(events, 'and')]; -} - -export function waitWords(task: JsonObject): Phrases { - const duration = durationText(field(task, 'wait')); - return [duration === undefined ? 'waits for a time computed when it runs' : `waits ${duration}`]; -} - -export function emitWords(task: JsonObject): Phrases { - const type = literalText(field(objectAt(task, 'emit', 'event', 'with'), 'type')); - return type === undefined - ? ['publishes an event of a type computed when it runs to the playbook'] - : ['publishes the event ', named(type), ' to the playbook']; -} - -function statusWords(error: JsonObject): Phrases { - const status = field(error, 'status'); - return typeof status === 'number' ? [`, with status ${status}`] : []; -} - -export function raiseWords(task: JsonObject): Phrases { - const reference = field(objectAt(task, 'raise'), 'error'); - if (typeof reference === 'string') { - return ['raises the error ', named(reference), ' its document names under ', { code: 'use.errors' }]; - } - const error = objectAt(task, 'raise', 'error'); - const title = textField(error, 'title'); - const type = textField(error, 'type'); - if (title !== undefined) { - return ['raises the error ', { emphasis: title }, ...statusWords(error)]; - } - return type === undefined - ? ['raises an error'] - : ['raises an error of the type ', { code: type }, ...statusWords(error)]; -} - -export function runWords(task: JsonObject): Phrases { - const kind = runKindOf(task); - return [kind === undefined ? 'runs a process' : `runs a ${kind}`]; -} diff --git a/capabilities/coordination/src/steps/tried-tasks.test.ts b/capabilities/coordination/src/steps/tried-tasks.test.ts deleted file mode 100644 index 3fb62af62..000000000 --- a/capabilities/coordination/src/steps/tried-tasks.test.ts +++ /dev/null @@ -1,176 +0,0 @@ -import { describe, expect, it } from 'vitest'; - -import { doneIn, stepNumbersOf, type DoneSteps } from '../testing/done-steps.ts'; - -interface Example extends DoneSteps { - readonly tasks: readonly string[]; -} - -function drawnFrom({ tasks }: Example): DoneSteps { - return doneIn(tasks.join('\n')); -} - -function expectedOf({ diagram, steps }: Example): DoneSteps { - return { diagram, steps }; -} - -const recovered: Example = { - tasks: [ - ' - attempt:', - ' try:', - ' - review: { call: run_definition, with: { type: reasoning, name: review-brief } }', - ' - score: { call: run_definition, with: { type: computation, name: campaign-pace } }', - ' catch:', - ' errors: { with: { status: 503 } }', - ' retry: { delay: { seconds: 5 }, limit: { attempt: { count: 3 } } }', - ' do:', - ' - fallback: { set: { review: null } }', - ], - diagram: [ - ' subgraph g1["1. Try"]', - ' n2["2. Run review-brief"]', - ' n3["3. Run campaign-pace"]', - ' end', - ' n1(["By a call"])', - ' n4["4. Set review"]', - ' n1 --> n2', - ' n2 --> n3', - ' n2 -.->|"on error 503, after retries"| n4', - ' n3 -.->|"on error 503, after retries"| n4', - ], - steps: [ - '1. **attempt** tries steps 2 to 3; on an error with status 503, it tries again as its catch says, and once no retry is left, it does step 4.', - '2. **review** runs the reasoning function **review-brief**.', - '3. **score** runs the computation function **campaign-pace**.', - '4. **fallback** sets **review**.', - ], -}; - -const guarded: Example = { - tasks: [ - ' - guarded:', - ' try:', - " - risky: { call: run_definition, with: { type: reasoning, name: '${ $data.name }' } }", - ' catch:', - ' errors: { with: { type: https://on.auto/problems/unanswered, status: 410 } }', - ` when: '\${ $error.kind == "expired" }'`, - ` exceptWhen: '\${ $error.because == "notified" }'`, - ' retry: patient', - ' - after: { set: { done: true } }', - ], - diagram: [ - ' subgraph g1["1. Try"]', - ' n2["2. Run a definition"]', - ' end', - ' n1(["By a call"])', - ' n3["3. Set done"]', - ' n1 --> n2', - ' n2 --> n3', - ' n2 -.->|"on error 410, after retries"| n3', - ], - steps: [ - '1. **guarded** tries step 2; on an error whose type is `https://on.auto/problems/unanswered` and with status 410, when `$error.kind == "expired"`, except when `$error.because == "notified"`, it tries again under the retry policy **patient**, and once no retry is left, it hands on its input.', - '2. **risky** runs the definition its call names when it runs.', - '3. **after** sets **done**.', - ], -}; - -const nestedInALoop: Example = { - tasks: [ - ' - each:', - " for: { in: '${ $data.briefs }' }", - ' do:', - ' - attempt:', - ' try:', - " - review: { call: run_definition, with: { type: reasoning, name: review-brief, input: '${ $item }' } }", - ' catch:', - ' do:', - ' - skip: { set: { skipped: true } }', - ], - diagram: [ - ' subgraph g1["1. For each item of $data.briefs"]', - ' n3["4. Set skipped"]', - ' subgraph g2["2. Try"]', - ' n2["3. Run review-brief"]', - ' end', - ' end', - ' n1(["By a call"])', - ' n1 --> n2', - ' n2 -->|"next item"| n2', - ' n2 -.->|"on an error"| n3', - ' n3 -->|"next item"| n2', - ], - steps: [ - '1. **each** does steps 2 to 4 for each **item** of `$data.briefs`.', - '2. **attempt** tries step 3; on any error, it does step 4.', - '3. **review** runs the reasoning function **review-brief**.', - '4. **skip** sets **skipped**.', - ], -}; - -const aroundAGroup: Example = { - tasks: [ - ' - attempt:', - ' try:', - ' - both:', - ' do:', - ' - one: { set: { a: 1 } }', - ' - two: { set: { b: 2 } }', - ' catch:', - ' do:', - ' - recover: { set: { c: 3 } }', - ], - diagram: [ - ' subgraph g1["1. Try"]', - ' subgraph g2["2. In order"]', - ' n2["3. Set a"]', - ' n3["4. Set b"]', - ' end', - ' end', - ' n1(["By a call"])', - ' n4["5. Set c"]', - ' n1 --> n2', - ' n2 --> n3', - ' n2 -.->|"on an error"| n4', - ' n3 -.->|"on an error"| n4', - ], - steps: [ - '1. **attempt** tries steps 2 to 4; on any error, it does step 5.', - '2. **both** does steps 3 to 4 in order.', - '3. **one** sets **a**.', - '4. **two** sets **b**.', - '5. **recover** sets **c**.', - ], -}; - -describe('the diagram of a try', () => { - it('is a group, with its catch a dashed edge from each of its steps to the steps that recover', () => { - const done = drawnFrom(recovered); - - expect(done).toEqual(expectedOf(recovered)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); - - it('goes on with a dashed edge when its catch has no steps, and names the errors it catches', () => { - const done = drawnFrom(guarded); - - expect(done).toEqual(expectedOf(guarded)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); - -describe('the diagram of a task within another', () => { - it('nests its group within the group of the task around it, and the catch of a try in that group too', () => { - const done = drawnFrom(nestedInALoop); - - expect(done).toEqual(expectedOf(nestedInALoop)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); - - it('draws the catch of a try from each step of a group within it', () => { - const done = drawnFrom(aroundAGroup); - - expect(done).toEqual(expectedOf(aroundAGroup)); - expect(stepNumbersOf(done).drawn).toEqual(stepNumbersOf(done).listed); - }); -}); diff --git a/capabilities/coordination/src/testing/done-steps.ts b/capabilities/coordination/src/testing/done-steps.ts deleted file mode 100644 index 277d00a15..000000000 --- a/capabilities/coordination/src/testing/done-steps.ts +++ /dev/null @@ -1,41 +0,0 @@ -import { markdownOf, numberedList, sectionOf } from '@on.auto/definitions'; -import { field } from '@on.auto/workflow-engine'; - -import { taskDiagramOf } from '../steps/task-diagram.ts'; -import { workflow } from './workflows.ts'; - -export interface DoneSteps { - readonly diagram: readonly string[]; - readonly steps: readonly string[]; -} - -export interface StepNumbers { - readonly drawn: readonly number[]; - readonly listed: readonly number[]; -} - -const labelledStep = /^\s*(?:n\d+\W+|subgraph g\d+\[")(?\d+)\. /u; - -export function doneIn(tasks: string, entries: readonly string[] = ['By a call']): DoneSteps { - const { diagram, steps } = taskDiagramOf(field(workflow(`do:\n${tasks}`), 'do'), entries); - const parts = markdownOf({ - title: 'flow', - opening: [], - body: [], - sections: [sectionOf('What it does', diagram, numberedList(steps))], - appendix: [], - }).split('\n\n'); - return { diagram: String(parts[3]).split('\n').slice(2, -1), steps: String(parts[4]).trimEnd().split('\n') }; -} - -function stepOf(line: string): readonly number[] { - const step = labelledStep.exec(line)?.groups?.['step']; - return step === undefined ? [] : [Number(step)]; -} - -export function stepNumbersOf({ diagram, steps }: DoneSteps): StepNumbers { - return { - drawn: diagram.flatMap((line) => stepOf(line)).toSorted((left, right) => left - right), - listed: steps.map((item) => Number(item.slice(0, item.indexOf('.')))), - }; -} diff --git a/capabilities/interaction/src/capability/definition-reading.ts b/capabilities/interaction/src/capability/definition-reading.ts index 4af032dff..ddc90a5e9 100644 --- a/capabilities/interaction/src/capability/definition-reading.ts +++ b/capabilities/interaction/src/capability/definition-reading.ts @@ -1,4 +1,4 @@ -import type { DefinitionSummary, ToldParts } from '@on.auto/definitions'; +import type { DefinitionSummary } from '@on.auto/definitions'; import { issueText } from '@on.auto/definitions/document'; import { renderClock } from '@on.auto/definitions/template'; import { InvalidInput } from '@on.auto/operations'; @@ -6,8 +6,6 @@ import { Effect, Result, type Schema } from 'effect'; import { parseInteractionDocument } from '../document/document-parsing.ts'; import type { InteractionFunctionDefinitionDocument } from '../document/interaction-document.ts'; -import { callOverview } from '../overview/call-overview.ts'; -import { requestOverview } from '../overview/request-overview.ts'; export function parse(source: string): Effect.Effect { return Effect.flatMap(renderClock, (clock) => @@ -35,16 +33,11 @@ function detailsOf(document: InteractionFunctionDefinitionDocument): Schema.Json } export function summarize(document: InteractionFunctionDefinitionDocument): DefinitionSummary { - const { description, about, input, output } = document; + const { description, input, output } = document; return { ...(description === undefined ? {} : { description }), - ...(about === undefined ? {} : { about }), ...(input.schema === undefined ? {} : { inputSchema: input.schema.document }), ...(output.schema === undefined ? {} : { outputSchema: output.schema.document }), details: detailsOf(document), }; } - -export function overview(document: InteractionFunctionDefinitionDocument): ToldParts { - return document.shape === 'call' ? callOverview(document) : requestOverview(document); -} diff --git a/capabilities/interaction/src/capability/interaction-function.ts b/capabilities/interaction/src/capability/interaction-function.ts index 0214de010..c9d3983bd 100644 --- a/capabilities/interaction/src/capability/interaction-function.ts +++ b/capabilities/interaction/src/capability/interaction-function.ts @@ -10,7 +10,7 @@ import type { InteractionFunctionDefinitionDocument } from '../document/interact import { interactionFacts } from '../facts/declared-facts.ts'; import { asksLater, requestRun } from '../run/interaction-run.ts'; import type { InteractionPorts } from '../run/request-reach.ts'; -import { overview, parse, summarize } from './definition-reading.ts'; +import { parse, summarize } from './definition-reading.ts'; import { interactionType } from './interaction-type.ts'; import { describeAnswer, interactionRunWords } from './interaction-words.ts'; import { cancelledRequest } from './request-cancels.ts'; @@ -40,7 +40,6 @@ export function makeInteractionFunctionAdapter(ports: InteractionPorts): Capabil mediaType: 'text/markdown', parse, summarize, - overview, run: (document, input, context) => document.shape === 'call' ? called(document, input, context) : asked(document, input, context), whenCancelled: 'finish', diff --git a/capabilities/interaction/src/document/call-parts.ts b/capabilities/interaction/src/document/call-parts.ts index b7ee2ede1..6315a8748 100644 --- a/capabilities/interaction/src/document/call-parts.ts +++ b/capabilities/interaction/src/document/call-parts.ts @@ -66,7 +66,6 @@ export function callFrom( : Result.map(Result.all({ input, output: answered }), (contracts: Pick) => ({ shape: 'call', ...(written.description === undefined ? {} : { description: written.description }), - ...(written.about === undefined ? {} : { about: written.about }), call, ...contracts, })); diff --git a/capabilities/interaction/src/document/document-parsing.test.ts b/capabilities/interaction/src/document/document-parsing.test.ts index e128e08a7..57a61ef36 100644 --- a/capabilities/interaction/src/document/document-parsing.test.ts +++ b/capabilities/interaction/src/document/document-parsing.test.ts @@ -3,7 +3,7 @@ import { standingRenderClock } from '@on.auto/definitions/testing'; import { Result } from 'effect'; import { describe, expect, it } from 'vitest'; -import { approvalDocument, callDocument, notificationDocument } from '../testing/index.ts'; +import { approvalDocument, notificationDocument } from '../testing/index.ts'; import { requestDocumentOf } from '../testing/route-documents.ts'; import { parseInteractionDocument } from './document-parsing.ts'; @@ -18,7 +18,7 @@ function documentWith(frontMatter: readonly string[], body = 'Approve {{ input.c const essentials = ["to: '{{ input.owner }}'", 'expires: P2D']; -const keys = 'description, about, call, to, from, expires, deliver, replies, input, output, reply'; +const keys = 'description, call, to, from, expires, deliver, replies, input, output, reply'; describe('an interaction function definition', () => { it('reads its party, expiry, schemas and message, and waits in the inbox without deliver', () => { @@ -154,26 +154,3 @@ describe('the refusals of the form of a definition', () => { ]); }); }); - -describe('the about of an interaction function definition', () => { - const about = ['about: |', ' The campaign owner approves each brief before it goes to a client.']; - - it('is kept as its author wrote it, for a function that asks a person and for one that asks a system', () => { - expect([ - requestDocumentOf(documentWith([...essentials, ...about])).about, - Result.getOrThrow(parseInteractionDocument(callDocument({ front: about }), standingRenderClock)).about, - ]).toEqual([ - 'The campaign owner approves each brief before it goes to a client.\n', - 'The campaign owner approves each brief before it goes to a client.\n', - ]); - }); - - it('takes no heading, nor does the description, each refused at its line', () => { - const frontMatter = ["description: '# Approve a brief'", ...essentials, ...about, ' ## When it runs']; - - expect(problemsOf(documentWith(frontMatter))).toEqual([ - 'Line 2, /description: description takes no heading, since an overview writes its own: the line that begins “# Approve a brief” makes one, with # before its words or as a line of = or - under the words above it', - 'Line 7, /about: about takes no heading, since an overview writes its own: this line makes one, with # before its words or as a line of = or - under the words above it', - ]); - }); -}); diff --git a/capabilities/interaction/src/document/front-matter.ts b/capabilities/interaction/src/document/front-matter.ts index 38f43fcd1..09fb9b8d3 100644 --- a/capabilities/interaction/src/document/front-matter.ts +++ b/capabilities/interaction/src/document/front-matter.ts @@ -1,9 +1,4 @@ -import { - AboutSchema, - DescriptionSchema, - type FrontMatterSection, - type FrontMatterShape, -} from '@on.auto/definitions/document'; +import { DescriptionSchema, type FrontMatterSection, type FrontMatterShape } from '@on.auto/definitions/document'; import { Schema } from 'effect'; import { WrittenRuleSchema } from '../replies/reply-rule.ts'; @@ -21,7 +16,6 @@ const SectionSchema = Schema.Struct({ schema: Schema.optionalKey(Schema.JsonObje const FrontMatterSchema = Schema.Struct({ description: Schema.optionalKey(DescriptionSchema), - about: Schema.optionalKey(AboutSchema), call: Schema.optionalKey(CallBlockSchema), to: Schema.optionalKey(Schema.String), from: Schema.optionalKey(Schema.String), diff --git a/capabilities/interaction/src/document/interaction-document.ts b/capabilities/interaction/src/document/interaction-document.ts index 06e39ace6..837b8c60b 100644 --- a/capabilities/interaction/src/document/interaction-document.ts +++ b/capabilities/interaction/src/document/interaction-document.ts @@ -8,7 +8,6 @@ import type { CallBlock } from '../tool-blocks/tool-block-schemas.ts'; interface WrittenFunction { readonly description?: string; - readonly about?: string; readonly input: ValueContract; } diff --git a/capabilities/interaction/src/document/request-parts.ts b/capabilities/interaction/src/document/request-parts.ts index 0c4456e05..8a9310cd9 100644 --- a/capabilities/interaction/src/document/request-parts.ts +++ b/capabilities/interaction/src/document/request-parts.ts @@ -64,7 +64,6 @@ function documentOfParts(written: InteractionFrontMatter, expires: string, done: return { shape: 'request', ...(written.description === undefined ? {} : { description: written.description }), - ...(written.about === undefined ? {} : { about: written.about }), ...(route === null ? {} : { route }), to, ...(from === null ? {} : { from }), diff --git a/capabilities/interaction/src/overview/approve-report.overview.md b/capabilities/interaction/src/overview/approve-report.overview.md deleted file mode 100644 index 5e8b5dffb..000000000 --- a/capabilities/interaction/src/overview/approve-report.overview.md +++ /dev/null @@ -1,62 +0,0 @@ -# approve-report - -Sends the draft report to the approver in the team chat and waits for a reply of approve or reject in the thread. - -```mermaid -flowchart LR - n1["The approver, the month and the draft report"] - n2["A message to the approver through gas"] - n3["Waits up to 7 days for a reply"] - n4{"The first word of the reply"} - n5(["approve"]) - n6(["reject"]) - n7(["unanswered"]) - n1 --> n2 - n2 --> n3 - n3 --> n4 - n4 -->|"approve, approved, yes, ok, okay, lgtm or ship"| n5 - n4 -->|"reject, rejected, no or nope"| n6 - n4 -.->|"any other word, told how to answer"| n3 - n3 -.->|"no reply"| n7 -``` - -It is given the approver, the month and the draft report. It sends the approver this message through **gas**, with the run's values in place of the words in brackets: - -> :wave: Draft website report for *\[the month\]* is ready for your okay before it goes to #random. -> -> \[the draft report\] -> -> Reply *in this thread* with *approve* to log it in the tracker sheet and post it, or *reject* (optionally with a note). - -It waits up to 7 days for a reply to that message, reading the replies at least 15 seconds apart. A reply whose first word is approve, approved, yes, ok, okay, lgtm or ship gives the verdict approve; one whose first word is reject, rejected, no or nope gives reject; whatever follows the first word is the note. A reply that begins with any other word is not taken, and the person is told how to answer. If nobody answers within 7 days, the request ends unanswered. - -It answers with the verdict, one of approve and reject, and the note, when there is one. - -## Used by - -- The workflow **Monthly website report** calls it in step 6, **Send draft**. - -## Details - -### Input - -| Field | Required | What it is | -| --- | --- | --- | -| **approver** | yes | Chat user id of the approver. Text. | -| **label** | yes | The month, such as April 2026. Text. | -| **draft** | yes | The draft report. Text, at most 6,000 characters. | - -### Answer - -| Field | Required | What it is | -| --- | --- | --- | -| **verdict** | yes | One of: approve, reject. | -| **note** | no | Text, at most 2,000 characters. | - -### Where - -To `{{ input.approver }}`. Sent through the tool `execute` of the server **gas**; the replies read with the tool `execute` of the server **gas**, at least `PT15S` apart; expires after `P7D`. - -### Used by - -- `monthly-website-report`, a workflow diff --git a/capabilities/interaction/src/overview/call-overview.ts b/capabilities/interaction/src/overview/call-overview.ts deleted file mode 100644 index a3594462e..000000000 --- a/capabilities/interaction/src/overview/call-overview.ts +++ /dev/null @@ -1,13 +0,0 @@ -import { callSection, callSentence, inputDetails, outputDetails, type ToldParts } from '@on.auto/definitions'; - -import type { CallDocument } from '../document/interaction-document.ts'; - -export function callOverview({ call, input, output }: CallDocument): ToldParts { - const { server, tool, read } = call; - const given = input.schema?.document; - const answered = output.schema.document; - return { - body: [callSentence({ input: given, server, tool, read, output: answered })], - appendix: [inputDetails(given), outputDetails(answered), callSection({ server, tool, read, arguments: call.with })], - }; -} diff --git a/capabilities/interaction/src/overview/interaction-overview.test.ts b/capabilities/interaction/src/overview/interaction-overview.test.ts deleted file mode 100644 index 78cc9e47c..000000000 --- a/capabilities/interaction/src/overview/interaction-overview.test.ts +++ /dev/null @@ -1,158 +0,0 @@ -import { readFileSync } from 'node:fs'; - -import { markdownOf, type Overview } from '@on.auto/definitions'; -import { onTheStandingClock, overviewAsRead } from '@on.auto/definitions/testing'; -import { Effect } from 'effect'; -import { describe, expect, it } from 'vitest'; - -import { callDocument, interactionHarness } from '../testing/index.ts'; - -const { capability } = interactionHarness(); - -function examplesOf(page: string): readonly string[] { - const text = readFileSync(new URL(`../../../../docs/reference/${page}`, import.meta.url), 'utf8'); - return [...text.matchAll(/```markdown\n([\s\S]*?)```/gu)].map(([, body = '']: readonly string[]) => body); -} - -function goldenOf(file: string): string { - return readFileSync(new URL(file, import.meta.url), 'utf8'); -} - -function composedOf(source: string, name: string): Overview { - return Effect.runSync(onTheStandingClock(capability.prepare(source))).overview(name); -} - -function overviewOf(source: string, name = 'read-thread'): string { - return markdownOf(composedOf(source, name)); -} - -const [inboxExample = '', chatExample = ''] = examplesOf('interaction-format.md'); - -const callExample = examplesOf('computation-format.md').find((example) => example.includes('\ncall:\n')) ?? ''; - -const approveReport = [ - '---', - 'description: Sends the draft report to the approver in the team chat and waits for a reply of approve or reject in the thread', - "to: '{{ input.approver }}'", - 'expires: P7D', - 'deliver:', - ' server: gas', - ' tool: execute', - ' with:', - ' function: postForApproval', - " parameters: ['{{ to }}', '{{ message }}']", - ' sent:', - ' conversation: /channel', - ' id: /ts', - 'replies:', - ' tool: execute', - ' with:', - ' function: repliesSince', - ` parameters: ['{{ sent.conversation }}', '{{ sent.id }}', '{{ since | default: "0" }}']`, - ' read:', - ' list: /replies', - ' order: oldest_first', - ' each: { id: /ts, sender: /user, text: /text }', - ' wait: PT15S', - ' tell:', - ' with:', - ' function: replyInThread', - " parameters: ['{{ sent.conversation }}', '{{ sent.id }}', '{{ message }}']", - 'reply:', - ' verdict: { from: word, words: { approve: [approved, yes, ok, okay, lgtm, ship], reject: [rejected, no, nope] } }', - ' note: rest', - 'input:', - ' schema:', - ' type: object', - ' required: [approver, label, draft]', - ' properties:', - " approver: { type: string, description: 'Chat user id of the approver' }", - " label: { type: string, description: 'The month, such as April 2026' }", - " draft: { type: string, maxLength: 6000, description: 'The draft report' }", - 'output:', - ' schema:', - ' type: object', - ' required: [verdict]', - ' properties:', - ' verdict: { type: string, enum: [approve, reject] }', - ' note: { type: string, maxLength: 2000 }', - '---', - ':wave: Draft website report for *{{ input.label }}* is ready for your okay before it goes to #random.', - '', - '{{ input.draft }}', - '', - 'Reply *in this thread* with *approve* to log it in the tracker sheet and post it, or *reject* (optionally with a note).', -].join('\n'); - -describe('the overviews of the examples of the reference of interaction functions', () => { - it('are, byte for byte, the overviews pinned beside this test, of a request to the inbox and through a chat', () => { - expect(overviewOf(inboxExample, 'approve-brief')).toBe(goldenOf('reference-inbox.overview.md')); - expect(overviewOf(chatExample, 'approve-draft')).toBe(goldenOf('reference-chat.overview.md')); - }); - - it('is, byte for byte for the function that asks a system, the overview pinned beside this test', () => { - expect(overviewOf(callExample, 'read-campaign-costs')).toBe(goldenOf('reference-call.overview.md')); - }); -}); - -describe('the overview of an interaction function that asks a person and reads the replies to its message', () => { - it('draws the reply, quotes the message with its values in brackets and tells what happens, as read the overview pinned beside this test', () => { - const caller = { - type: 'workflow', - name: 'monthly-website-report', - details: { - title: 'Monthly website report', - calls: [{ type: 'interaction', name: 'approve-report', steps: [{ number: 6, name: 'send-draft' }] }], - }, - }; - const overview = composedOf(approveReport, 'approve-report'); - - expect(overviewAsRead({ type: 'interaction', name: 'approve-report', overview }, [caller])).toBe( - goldenOf('approve-report.overview.md'), - ); - }); -}); - -describe('the overview of an interaction function that asks a system', () => { - it('names the server and the tool it asks, the input it is given and where in the answer it reads, in one paragraph', () => { - const source = callDocument({ read: '/data/items~1all~0x' }); - - expect(composedOf(source, 'read-thread').body).toHaveLength(1); - expect(overviewOf(source).split('\n\n')[2]).toBe( - 'It is given the channel and the thread and asks **chat** with its tool thread, once per run. It answers with what it answered under data, then items/all~x: a list of records.', - ); - }); - - it('answers with the whole answer without read, and writes the arguments of a call that reads no input as written', () => { - const overview = overviewOf(callDocument({ read: null, input: [], with: [' limit: 100'] })); - - expect(overview).toContain( - 'It is given any input and asks **chat** with its tool thread, once per run. It answers with the whole of what the tool answered:', - ); - expect(overview).toContain( - '### Call\n\nIt calls the tool `thread` of the server **chat**, its whole answer read, with these arguments, as written:\n\n- `limit`: `100`\n', - ); - }); -}); - -describe('the overview of an interaction function whose document has an about', () => { - it('writes the about as its author wrote it, after the description and before what it is given', () => { - const about = [ - 'about: |', - ' The campaign team reads what a thread said before it plans the next brief.', - '', - ' A workflow asks it once a thread is closed.', - ]; - - expect( - overviewOf(callDocument({ front: about })) - .split('\n\n') - .slice(0, 4), - ).toEqual([ - '# read-thread', - 'Read the replies of a thread in the team chat, oldest first.', - 'The campaign team reads what a thread said before it plans the next brief.', - 'A workflow asks it once a thread is closed.', - ]); - }); -}); diff --git a/capabilities/interaction/src/overview/reference-call.overview.md b/capabilities/interaction/src/overview/reference-call.overview.md deleted file mode 100644 index 9a37eb8f2..000000000 --- a/capabilities/interaction/src/overview/reference-call.overview.md +++ /dev/null @@ -1,23 +0,0 @@ -# read-campaign-costs - -Reads the cost rows of every campaign for a month. - -It is given the month and asks **ads** with its tool campaign-costs, once per run. It answers with what it answered under rows: a list of records. - -## Details - -### Input - -| Field | Required | What it is | -| --- | --- | --- | -| **month** | yes | Text. | - -### Output - -A list of records with: **campaign** (text), **cost_cents** (a whole number), **budget_cents** (a whole number). - -### Call - -It calls the tool `campaign-costs` of the server **ads**, its answer read at `/rows`, with these arguments, as written: - -- `month`: `{{ input.month }}` diff --git a/capabilities/interaction/src/overview/reference-chat.overview.md b/capabilities/interaction/src/overview/reference-chat.overview.md deleted file mode 100644 index e13111308..000000000 --- a/capabilities/interaction/src/overview/reference-chat.overview.md +++ /dev/null @@ -1,53 +0,0 @@ -# approve-draft - -Send the draft to its owner in the chat and take their approval. - -```mermaid -flowchart LR - n1["The owner and the draft"] - n2["A message to the owner through chat"] - n3["Waits up to 2 days for a reply"] - n4{"The first word of the reply"} - n5(["approve"]) - n6(["revise"]) - n7(["unanswered"]) - n1 --> n2 - n2 --> n3 - n3 --> n4 - n4 -->|"approve"| n5 - n4 -->|"revise"| n6 - n4 -.->|"any other word, told how to answer"| n3 - n3 -.->|"no reply"| n7 -``` - -It is given the owner and the draft. It sends the owner this message through **chat**, with the run's values in place of the words in brackets: - -> Here is the draft: -> -> \[the draft\] -> -> Reply **approve**, or **revise** with a note of what to change. - -It waits up to 2 days for a reply to that message, reading the replies at least 1 minute apart. A reply whose first word is approve gives the verdict approve; one whose first word is revise gives revise; whatever follows the first word is the note. A reply that begins with any other word is not taken, and the person is told how to answer. If nobody answers within 2 days, the request ends unanswered. - -It answers with the verdict, one of approve and revise, and the note, when there is one. - -## Details - -### Input - -| Field | Required | What it is | -| --- | --- | --- | -| **owner** | yes | Text. | -| **draft** | yes | Text, at most 4,000 characters. | - -### Answer - -| Field | Required | What it is | -| --- | --- | --- | -| **verdict** | yes | One of: approve, revise. | -| **note** | no | Text, at most 2,000 characters. | - -### Where - -To `{{ input.owner }}`. Sent through the tool `post_message` of the server **chat**; the replies read with the tool `thread_replies` of the server **chat**, at least `PT1M` apart; expires after `P2D`. diff --git a/capabilities/interaction/src/overview/reference-inbox.overview.md b/capabilities/interaction/src/overview/reference-inbox.overview.md deleted file mode 100644 index 1bf334c4e..000000000 --- a/capabilities/interaction/src/overview/reference-inbox.overview.md +++ /dev/null @@ -1,35 +0,0 @@ -# approve-brief - -Asks the campaign owner to approve a reviewed brief. - -It is given the owner and the review. It sends the owner this message, with the run's values in place of the words in brackets: - -> The review of your campaign brief is ready: -> -> \[the review\] -> -> Approve the brief as it stands, or ask for a revision. - -It leaves the message in the playbook's inbox, where whoever may write to the playbook answers it. If nobody answers within 2 days, the request ends unanswered. - -It answers with the verdict, one of approve and revise, and the note, when there is one. - -## Details - -### Input - -| Field | Required | What it is | -| --- | --- | --- | -| **owner** | yes | Text. | -| **review** | yes | Text. | - -### Answer - -| Field | Required | What it is | -| --- | --- | --- | -| **verdict** | yes | One of: approve, revise. | -| **note** | no | Text, at most 2,000 characters. | - -### Where - -To `{{ input.owner }}`. Left in the playbook's inbox; expires after `P2D`. diff --git a/capabilities/interaction/src/overview/request-overview.test.ts b/capabilities/interaction/src/overview/request-overview.test.ts deleted file mode 100644 index c6e99424d..000000000 --- a/capabilities/interaction/src/overview/request-overview.test.ts +++ /dev/null @@ -1,163 +0,0 @@ -import { markdownOf } from '@on.auto/definitions'; -import { onTheStandingClock } from '@on.auto/definitions/testing'; -import { Effect } from 'effect'; -import { describe, expect, it } from 'vitest'; - -import { - approvalDocument, - chatDelivery, - interactionHarness, - notificationDocument, - threadDocument, - threadReplies, -} from '../testing/index.ts'; - -const { capability } = interactionHarness(); - -function overviewOf(source: string): string { - return markdownOf(Effect.runSync(onTheStandingClock(capability.prepare(source))).overview('approve-brief')); -} - -function bodyOf(source: string): readonly string[] { - const [story = ''] = overviewOf(source).split('\n\n## Details\n\n'); - return story.split('\n\n').slice(2); -} - -function leadOf(source: string): string { - return bodyOf(source).find((block) => block.startsWith('It is given')) ?? ''; -} - -function leadTo(to: string): string { - return leadOf(approvalDocument().replace("to: '{{ input.owner }}'", to)); -} - -function detailsIn(overview: string, heading: string): string { - return overview.split(`\n\n### ${heading}\n\n`)[1]?.split('\n\n### ')[0] ?? ''; -} - -function waitOf(expires: string): string { - return bodyOf(approvalDocument([], expires)).at(-2) ?? ''; -} - -const choices = [ - 'output:', - ' schema:', - ' type: object', - ' required: [decision]', - ' properties:', - ' decision: { type: string, enum: [approve, revise, skip] }', - ' note: { type: string }', - ' said: { type: string }', - 'reply:', - ' decision: { from: word, words: { approve: [yes], skip: [later, pass] } }', - ' note: rest', - ' said: text', -]; - -const counted = [ - 'output:', - ' schema:', - ' type: object', - ' required: [count]', - ' properties:', - ' count: { type: integer }', -]; - -describe('how long a request waits, in the overview', () => { - it('says the expiry in days, hours, minutes and seconds, to the nearest second, for a request left in the inbox', () => { - expect(['P2D', 'PT1M', 'PT4H', 'P1DT12H30M', 'PT90.5S', 'P1W'].map((expires) => waitOf(expires))).toEqual( - ['2 days', '1 minute', '4 hours', '1 day, 12 hours, and 30 minutes', '1 minute and 31 seconds', '7 days'].map( - (words) => - `It leaves the message in the playbook's inbox, where whoever may write to the playbook answers it. If nobody answers within ${words}, the request ends unanswered.`, - ), - ); - }); - - it('says of a notification that it waits for nothing, takes no answer and draws nothing, with its expiry under Details', () => { - const inbox = overviewOf(notificationDocument()); - const sent = overviewOf(notificationDocument(chatDelivery)); - - expect([bodyOf(notificationDocument()).at(-1), bodyOf(notificationDocument(chatDelivery)).at(-1)]).toEqual([ - "It leaves the message in the playbook's inbox, where whoever may write to the playbook reads it, and waits for nothing.", - 'It sends the message and waits for nothing.', - ]); - expect([detailsIn(inbox, 'Answer'), detailsIn(sent, 'Where')]).toEqual([ - 'None: it is a notification, which takes no answer and succeeds once its request is delivered.', - 'To `{{ input.owner }}`. Sent through the tool `post_message` of the server **chat**; expires after `PT1H`.\n', - ]); - expect([inbox, sent].map((overview) => [overview.includes('mermaid'), overview.includes('It answers')])).toEqual([ - [false, false], - [false, false], - ]); - }); - - it('says that a request sent through a tool whose replies it does not read waits for the answer of whoever may write', () => { - expect([ - bodyOf(approvalDocument(chatDelivery)).at(-2), - bodyOf(approvalDocument(chatDelivery, 'P2D', counted)), - ]).toEqual([ - 'It waits up to 2 days for the answer, which whoever may write to the playbook gives. If nobody answers within 2 days, the request ends unanswered.', - [ - "It is given the campaign and the owner. It sends the owner this message through **chat**, with the run's values in place of the words in brackets:", - '> Please review the brief for \\[the campaign\\].', - 'It waits up to 2 days for the answer, which whoever may write to the playbook gives. If nobody answers within 2 days, the request ends unanswered.', - 'It answers with the count.', - ], - ]); - }); -}); - -describe('what a request is given and whom it asks, in the overview', () => { - it('says the fields it is given, then the party the value of its to means, before the message it quotes', () => { - expect(bodyOf(approvalDocument()).slice(0, 2)).toEqual([ - "It is given the campaign and the owner. It sends the owner this message, with the run's values in place of the words in brackets:", - '> Please review the brief for \\[the campaign\\].', - ]); - }); - - it('names a party written as text on one line as it is written, and any other party by the party named under Details', () => { - expect([ - leadTo("to: '#approvals'"), - leadTo("to: '{% if input.urgent %}on-call{% else %}{{ input.owner }}{% endif %}'"), - leadTo('to: "{{ input.owner }}\\n{{ input.campaign }}"'), - leadTo("to: '{{ input.owner }} {{ input.campaign }}'"), - ]).toEqual([ - "It is given the campaign and the owner. It sends #approvals this message, with the run's values in place of the words in brackets:", - "It is given the campaign and the owner. It sends the party named under Details this message, with the run's values in place of the words in brackets:", - "It is given the campaign and the owner. It sends the party named under Details this message, with the run's values in place of the words in brackets:", - "It is given the campaign and the owner. It sends the party named under Details this message, with the run's values in place of the words in brackets:", - ]); - }); - - it('writes the party and the answerer as their document writes them under Details', () => { - expect(detailsIn(overviewOf(threadDocument()), 'Where')).toBe( - 'To `#approvals-{{ input.team }}`, answered by `{{ input.owner }}`. Sent through the tool `post_message` of the server **chat**; the replies read with the tool `thread_replies` of the server **chat**, at least `PT1M` apart; expires after `P2D`.\n', - ); - }); -}); - -describe('the replies of a request, in the overview', () => { - it('say how a reply gives each part of the answer, what it reads first and that a reply of another word is not taken', () => { - const source = threadDocument({ rule: [] }).replace(/output:\n[\s\S]*maxLength: 2000 \}/u, choices.join('\n')); - - expect(bodyOf(source).slice(-2)).toEqual([ - 'It waits up to 2 days for a reply to that message, reading the replies at least 1 minute apart. A reply whose first word is approve or yes gives the decision approve; one whose first word is revise gives revise; one whose first word is skip, later or pass gives skip; whatever follows the first word is the note; the whole reply is the said. A reply that begins with any other word is not taken, and the person is told how to answer. If nobody answers within 2 days, the request ends unanswered.', - 'It answers with the decision, one of approve, revise and skip, the note, when there is one, and the said, when there is one.', - ]); - }); - - it('say that a reply of another word is not taken, without telling the person where the function does not', () => { - const source = threadDocument({ replies: threadReplies.slice(0, -5) }); - - expect(bodyOf(source).at(-2)).toMatch(/ A reply that begins with any other word is not taken\. If nobody/u); - expect(overviewOf(source)).toContain(' n4 -.->|"any other word"| n3\n'); - }); - - it('draw no reply and refuse no word of a rule that reads no first word', () => { - const overview = overviewOf(threadDocument({ rule: ['reply:', ' note: text'] })); - - expect(overview).not.toContain('mermaid'); - expect(overview).not.toContain('not taken'); - expect(overview).toContain('reading the replies at least 1 minute apart. The whole reply is the note. If nobody'); - }); -}); diff --git a/capabilities/interaction/src/overview/request-overview.ts b/capabilities/interaction/src/overview/request-overview.ts deleted file mode 100644 index 061309ba9..000000000 --- a/capabilities/interaction/src/overview/request-overview.ts +++ /dev/null @@ -1,125 +0,0 @@ -import { - answerSection, - answerSentence, - answerWaitSentence, - inboxSentence, - inputDetails, - messageLead, - notTakenSentence, - notificationSentence, - paragraph, - replyDiagram, - replyRuleSentence, - replyWaitSentence, - unansweredSentence, - whereSection, - type OverviewBlock, - type Phrases, - type ToldParts, -} from '@on.auto/definitions'; -import { quotedTemplate } from '@on.auto/definitions/template'; -import type { Schema } from 'effect'; - -import { replyWaitMsOf } from '../conversations/cadence.ts'; -import type { RequestDocument } from '../document/interaction-document.ts'; -import type { ReplyRule } from '../replies/reply-rule.ts'; -import type { WrittenRoute } from '../tool-blocks/compiled-tool-block.ts'; -import type { Replies } from '../tool-blocks/tool-block-schemas.ts'; -import { partyOf } from './request-party.ts'; - -type Words = Readonly>; - -interface ReadReplies { - readonly route: WrittenRoute; - readonly replies: Replies; - readonly reply: ReplyRule; - readonly answer: Schema.JsonObject; -} - -function verdictsOf(reply: ReplyRule): Words | undefined { - for (const part of Object.values(reply)) { - if (part.from === 'word') { - return part.words; - } - } - return undefined; -} - -function readRepliesOf({ route, reply, output }: RequestDocument): ReadReplies | undefined { - const answer = output.schema?.document; - return route?.replies === undefined || reply === undefined || answer === undefined - ? undefined - : { route, replies: route.replies, reply, answer }; -} - -function readingSentences({ replies, reply, answer }: ReadReplies, expiresMs: number): readonly string[] { - return [ - replyWaitSentence(expiresMs, replyWaitMsOf(replies)), - replyRuleSentence(reply, answer), - ...(verdictsOf(reply) === undefined ? [] : [notTakenSentence(replies.tell !== undefined)]), - unansweredSentence(expiresMs), - ]; -} - -function waitingSentences(document: RequestDocument): readonly string[] { - const { route, output, expiresMs } = document; - const read = readRepliesOf(document); - if (read !== undefined) { - return readingSentences(read, expiresMs); - } - if (output.schema === undefined) { - return [route === undefined ? inboxSentence('nothing') : notificationSentence]; - } - return [route === undefined ? inboxSentence('answer') : answerWaitSentence(expiresMs), unansweredSentence(expiresMs)]; -} - -function diagramOf(document: RequestDocument, party: Phrases): readonly OverviewBlock[] { - const read = readRepliesOf(document); - const verdicts = read === undefined ? undefined : verdictsOf(read.reply); - if (read === undefined || verdicts === undefined) { - return []; - } - return [ - replyDiagram({ - input: document.input.schema?.document, - party, - server: read.route.deliver.server, - expiresMs: document.expiresMs, - verdicts, - told: read.replies.tell !== undefined, - }), - ]; -} - -function whereOf({ to, from, route, expires }: RequestDocument): ReturnType { - const replies = route?.replies; - return whereSection({ - to: to.text, - from: from?.text, - route: - route === undefined - ? undefined - : { - server: route.deliver.server, - tool: route.deliver.tool, - replies: replies === undefined ? undefined : { tool: replies.tool, wait: replies.wait }, - }, - expires, - }); -} - -export function requestOverview(document: RequestDocument): ToldParts { - const input = document.input.schema?.document; - const answer = document.output.schema?.document; - const party = partyOf(document.to, input); - return { - body: [ - ...diagramOf(document, party), - messageLead({ input, party, server: document.route?.deliver.server }), - ...quotedTemplate(document.message, input), - paragraph(waitingSentences(document).join(' ')), - ...(answer === undefined ? [] : [answerSentence(answer)]), - ], - appendix: [inputDetails(input), answerSection(answer), whereOf(document)], - }; -} diff --git a/capabilities/interaction/src/overview/request-party.ts b/capabilities/interaction/src/overview/request-party.ts deleted file mode 100644 index c2fabd05b..000000000 --- a/capabilities/interaction/src/overview/request-party.ts +++ /dev/null @@ -1,16 +0,0 @@ -import { partyNamedUnderDetails, valueMeaning, type Phrases } from '@on.auto/definitions'; -import type { ParsedTemplate } from '@on.auto/definitions/template'; -import type { Schema } from 'effect'; - -const oneOutput = /^\{\{[^{}]*\}\}$/u; - -const liquid = /\{[{%]/u; - -export function partyOf({ text, variables }: ParsedTemplate, inputSchema: Schema.JsonObject | undefined): Phrases { - const written = text.trim(); - const [only, ...others] = variables; - if (oneOutput.test(written) && only !== undefined && others.length === 0) { - return [valueMeaning(only.path, inputSchema)]; - } - return liquid.test(written) || written.includes('\n') ? [partyNamedUnderDetails] : [written]; -} diff --git a/capabilities/interaction/src/testing/index.ts b/capabilities/interaction/src/testing/index.ts index c7646700c..9b3ef40a1 100644 --- a/capabilities/interaction/src/testing/index.ts +++ b/capabilities/interaction/src/testing/index.ts @@ -14,7 +14,6 @@ export { notificationDocument, replyRuleLines, threadDocument, - threadReplies, type ThreadOptions, } from './documents.ts'; export { callDocument, repliesOutput, type CallOptions } from './call-documents.ts'; diff --git a/capabilities/prediction/measure/order-history.ts b/capabilities/prediction/measure/order-history.ts index a00f667e2..a87991573 100644 --- a/capabilities/prediction/measure/order-history.ts +++ b/capabilities/prediction/measure/order-history.ts @@ -1,4 +1,3 @@ -import { overviewTitled } from '@on.auto/definitions/testing'; import type { ProjectedFolders } from '@on.auto/workflow-host/testing'; import { Result } from 'effect'; @@ -94,7 +93,7 @@ export function savedIn(folders: ProjectedFolders, { name, source, document }: M [ { type: 'definition_created', - data: { content: { source, overview: overviewTitled(name), details: datasetDetailsOf(document) } }, + data: { content: { source, details: datasetDetailsOf(document) } }, }, ], { diff --git a/capabilities/prediction/src/capability/prediction-definitions.ts b/capabilities/prediction/src/capability/prediction-definitions.ts index 561c84160..36da2a291 100644 --- a/capabilities/prediction/src/capability/prediction-definitions.ts +++ b/capabilities/prediction/src/capability/prediction-definitions.ts @@ -53,11 +53,10 @@ function callsOf({ source, evaluation: { activate } }: PredictionFunctionDefinit } export function summarize(document: PredictionFunctionDefinitionDocument): DefinitionSummary { - const { description, about, schemas } = document; + const { description, schemas } = document; const calls = callsOf(document); return { ...(description === undefined ? {} : { description }), - ...(about === undefined ? {} : { about }), inputSchema: schemas.input, ...publishedOutputOf({ shape: seriesShape, diff --git a/capabilities/prediction/src/capability/prediction-function.ts b/capabilities/prediction/src/capability/prediction-function.ts index 51af944b7..58a15b6d3 100644 --- a/capabilities/prediction/src/capability/prediction-function.ts +++ b/capabilities/prediction/src/capability/prediction-function.ts @@ -7,7 +7,6 @@ import { import { predictionCheck } from '../checks/prediction-check.ts'; import { predictionFacts } from '../facts/prediction-facts.ts'; -import { predictionOverview } from '../overview/prediction-overview.ts'; import { predictionRunsOn } from '../run/prediction-runs.ts'; import { predictionStanding } from '../standing/prediction-standing.ts'; import { predictionBounds, predictionDefinitionType } from './prediction-bounds.ts'; @@ -31,7 +30,6 @@ export function makePredictionFunctionAdapter({ parse, check: predictionCheck(pool), summarize, - overview: predictionOverview, ...predictionRunsOn(ports), facts: predictionFacts, mostActive: mostFunctions, diff --git a/capabilities/prediction/src/dataset-testing/projected-datasets.ts b/capabilities/prediction/src/dataset-testing/projected-datasets.ts index bdf1d7205..1547624a8 100644 --- a/capabilities/prediction/src/dataset-testing/projected-datasets.ts +++ b/capabilities/prediction/src/dataset-testing/projected-datasets.ts @@ -1,4 +1,3 @@ -import { overviewTitled } from '@on.auto/definitions/testing'; import { filterSandboxOf } from '@on.auto/workflow-engine/dsl'; import { someFolder, type DatabaseSettings } from '@on.auto/workflow-host'; import { projectedFolders, type ProjectedFolders } from '@on.auto/workflow-host/testing'; @@ -52,11 +51,7 @@ function definitionsOf(folders: ProjectedFolders): Pick { 'Line 3: The definition has no brief: write it after the front matter', ]); expect(problemsOf(weeklyOrdersWith(['source:', 'input:\n schema: {}\nsource:']))).toEqual([ - 'Line 24, /input: input is not a key of the front matter; it takes description, about, predicts, data, evaluation, source, rules', + 'Line 24, /input: input is not a key of the front matter; it takes description, predicts, data, evaluation, source, rules', ]); expect(problemsOf(weeklyOrdersWith([' period: week', ' period: fortnight']))).toEqual([periodRefused]); }); }); - -const descriptionLine = 'description: Predicts the orders of each region one and four weeks ahead'; - -describe('the about of a prediction function document', () => { - it('is kept as its author wrote it', () => { - const about = `${descriptionLine}\nabout: |\n The sales team plans the stock of each region with it.`; - - expect(documentOf(weeklyOrdersWith([descriptionLine, about])).about).toBe( - 'The sales team plans the stock of each region with it.\n', - ); - }); - - it('takes no heading, nor does the description, each refused at its line', () => { - const written = "description: '# Predicts the orders'\nabout: |\n The sales team plans with it.\n ==="; - - expect(problemsOf(weeklyOrdersWith([descriptionLine, written]))).toEqual([ - 'Line 2, /description: description takes no heading, since an overview writes its own: the line that begins “# Predicts the orders” makes one, with # before its words or as a line of = or - under the words above it', - 'Line 5, /about: about takes no heading, since an overview writes its own: this line makes one, with # before its words or as a line of = or - under the words above it', - ]); - }); -}); diff --git a/capabilities/prediction/src/document/document-parsing.ts b/capabilities/prediction/src/document/document-parsing.ts index eaad0f75c..8d524a639 100644 --- a/capabilities/prediction/src/document/document-parsing.ts +++ b/capabilities/prediction/src/document/document-parsing.ts @@ -23,7 +23,6 @@ function assembled( const { predicts, data, evaluation, source, rules, brief } = done; return { ...(front.description === undefined ? {} : { description: front.description }), - ...(front.about === undefined ? {} : { about: front.about }), predicts, data: data.data, dataSha256: dataDigestOf(data.data, predicts.period), diff --git a/capabilities/prediction/src/document/front-matter.ts b/capabilities/prediction/src/document/front-matter.ts index 1bcd8aea1..7ae6bc2b8 100644 --- a/capabilities/prediction/src/document/front-matter.ts +++ b/capabilities/prediction/src/document/front-matter.ts @@ -1,9 +1,4 @@ -import { - AboutSchema, - DescriptionSchema, - type FrontMatterSection, - type FrontMatterShape, -} from '@on.auto/definitions/document'; +import { DescriptionSchema, type FrontMatterSection, type FrontMatterShape } from '@on.auto/definitions/document'; import { Schema } from 'effect'; import { enclosings } from '../periods/enclosing.ts'; @@ -55,7 +50,6 @@ const SourceSchema = Schema.Struct({ const FrontMatterSchema = Schema.Struct({ description: Schema.optionalKey(DescriptionSchema), - about: Schema.optionalKey(AboutSchema), predicts: PredictsSchema, data: DataSchema, evaluation: EvaluationSchema, diff --git a/capabilities/prediction/src/document/prediction-document.ts b/capabilities/prediction/src/document/prediction-document.ts index eed4627f5..abd9fbd4c 100644 --- a/capabilities/prediction/src/document/prediction-document.ts +++ b/capabilities/prediction/src/document/prediction-document.ts @@ -84,7 +84,6 @@ export interface DerivedSchemas { export interface PredictionFunctionDefinitionDocument { readonly description?: string; - readonly about?: string; readonly predicts: Predicts; readonly data: DataPart; readonly dataSha256: string; diff --git a/capabilities/prediction/src/overview/data-words.ts b/capabilities/prediction/src/overview/data-words.ts deleted file mode 100644 index 181a0abc9..000000000 --- a/capabilities/prediction/src/overview/data-words.ts +++ /dev/null @@ -1,92 +0,0 @@ -import { - bulletList, - eventFilterPhrases, - named, - paragraph, - sectionOf, - type OverviewBlock, - type OverviewSection, - type Phrases, -} from '@on.auto/definitions'; - -import { - isProjected, - type Aggregation, - type Feature, - type Measure, - type PredictionFunctionDefinitionDocument, - type ProjectedData, -} from '../document/prediction-document.ts'; -import { askedPeriods, periodsOf } from './predicts-words.ts'; - -const measureWords: Readonly, string>> = { - sum: 'the sum of ', - last: 'the last value of ', - mean: 'the mean of ', -}; - -function measurePhrases({ of, as }: Measure): Phrases { - return of === undefined || as === 'count' ? ['the number of its events'] : [measureWords[as], { code: of }]; -} - -function givenPhrases(name: string, given: readonly string[], asked: string): Phrases { - return given.includes(name) ? [`, which a question gives for ${asked}`] : []; -} - -function featurePhrases({ name, ...measure }: Feature, given: readonly string[], asked: string): Phrases { - return [named(name), ', ', ...measurePhrases(measure), ...givenPhrases(name, given, asked)]; -} - -function timePhrases(time: string, period: string): Phrases { - return time === 'event' - ? [`the ${period}, from the time of each event`] - : [`the ${period}, from `, { code: time }, " in each event's data"]; -} - -function projectedBlocks( - { events, time, series, target, features, given }: ProjectedData, - { predicts }: PredictionFunctionDefinitionDocument, -): readonly OverviewBlock[] { - const asked = askedPeriods(predicts); - return [ - paragraph('It learns from the rows the playbook folds from the first event of its history on:'), - bulletList(events.map((filter) => ['each time ', ...eventFilterPhrases(filter)])), - paragraph(`Each row is one series and one ${predicts.period}, and holds:`), - bulletList([ - timePhrases(time, predicts.period), - ['the series, from ', { code: series }], - [named(predicts.target), ', ', ...measurePhrases(target)], - ...features.map((feature) => featurePhrases(feature, given, asked)), - ]), - ]; -} - -function uploadedBlocks( - dataset: string, - features: readonly string[], - { predicts, data }: PredictionFunctionDefinitionDocument, -): readonly OverviewBlock[] { - const asked = askedPeriods(predicts); - const opening = paragraph( - 'It learns from the dataset uploaded to the playbook as ', - { code: dataset }, - `, one row a series and a ${predicts.period}, with `, - named(predicts.target), - features.length === 0 ? ' alone.' : ' and then these columns:', - ); - return features.length === 0 - ? [opening] - : [opening, bulletList(features.map((name) => [named(name), ...givenPhrases(name, data.given, asked)]))]; -} - -export function learnedSection(document: PredictionFunctionDefinitionDocument): OverviewSection { - const { data, predicts } = document; - const blocks = isProjected(data) - ? projectedBlocks(data, document) - : uploadedBlocks(data.dataset, data.features, document); - return sectionOf( - 'What it learns from', - ...blocks, - paragraph(`A build reads the last ${periodsOf(predicts.period, data.window)}.`), - ); -} diff --git a/capabilities/prediction/src/overview/evaluation-words.ts b/capabilities/prediction/src/overview/evaluation-words.ts deleted file mode 100644 index 4902741ca..000000000 --- a/capabilities/prediction/src/overview/evaluation-words.ts +++ /dev/null @@ -1,69 +0,0 @@ -import { - listedPhrases, - named, - paragraph, - sectionOf, - type OverviewBlock, - type OverviewSection, -} from '@on.auto/definitions'; - -import { - isProjected, - type Baseline, - type Evaluation, - type PredictionFunctionDefinitionDocument, -} from '../document/prediction-document.ts'; -import { periodsOf } from './predicts-words.ts'; - -const metricWords: Readonly> = { - mase: 'its mean absolute scaled error', - winkler: 'the Winkler score of its interval', - quantile: 'the quantile loss of its percentiles', -}; - -const baselineWords: Readonly> = { - persistence: 'the last value carried forward', - 'seasonal-naive': 'the value one season earlier', -}; - -const share = new Intl.NumberFormat('en', { style: 'percent', maximumFractionDigits: 2 }); - -function shadowBlocks({ data, predicts, evaluation }: PredictionFunctionDefinitionDocument): readonly OverviewBlock[] { - const { shadowPeriods } = evaluation; - return shadowPeriods === 0 || !isProjected(data) - ? [] - : [ - paragraph( - `A version after the first runs beside the active one for ${periodsOf(predicts.accumulates ?? predicts.period, shadowPeriods)}, and takes its place only when it predicts no worse.`, - ), - ]; -} - -function activationBlock({ activate }: Evaluation): OverviewBlock { - return activate === 'automatic' - ? paragraph('A version that passes becomes the active one on its own.') - : paragraph( - 'A version that passes becomes the active one once it is approved through the interaction function ', - named(activate.approved), - '.', - ); -} - -export function goingLiveSection(document: PredictionFunctionDefinitionDocument): OverviewSection { - const { evaluation, predicts } = document; - const { holdout, metric, coverageFloor, baselines, confidence } = evaluation; - const beaten = baselines.map((baseline) => [named(baseline), ` (${baselineWords[baseline]})`]); - return sectionOf( - 'How a version goes live', - paragraph( - `The playbook holds back the last ${periodsOf(predicts.period, holdout)} of the window from the trainer, and scores each candidate on them by ${metricWords[metric]}.`, - ), - paragraph( - `A candidate passes when its interval holds at least ${share.format(coverageFloor)} of the truths and it beats `, - ...listedPhrases(beaten, 'and'), - ` with ${share.format(confidence)} confidence; the best that passes becomes the next version.`, - ), - ...shadowBlocks(document), - activationBlock(evaluation), - ); -} diff --git a/capabilities/prediction/src/overview/prediction-overview.test.ts b/capabilities/prediction/src/overview/prediction-overview.test.ts deleted file mode 100644 index ed9fdbb23..000000000 --- a/capabilities/prediction/src/overview/prediction-overview.test.ts +++ /dev/null @@ -1,172 +0,0 @@ -import { readFileSync } from 'node:fs'; - -import { markdownOf } from '@on.auto/definitions'; -import { sectionIn } from '@on.auto/definitions/testing'; -import { describe, expect, it } from 'vitest'; - -import { dailyTotalsWith, dailyVisitsWith, uploadedDailyVisits } from '../testing/daily-documents.ts'; -import { predictionRuns } from '../testing/prediction-runs.ts'; -import { weeklyOrdersLines, weeklyOrdersWith } from '../testing/weekly-orders.ts'; - -const page = readFileSync(new URL('../../../../docs/reference/prediction-format.md', import.meta.url), 'utf8'); - -const [example = '', , dataPart = ''] = [...page.matchAll(/```(?:markdown|text|yaml)\n([\s\S]*?)```/gu)].map( - ([, body = '']: readonly string[]) => body, -); - -const sha256 = '3f2ac1d0e4b5a69788796a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d0e9f8a7b6c5d'; - -function goldenOf(name: string): string { - return readFileSync(new URL(name, import.meta.url), 'utf8'); -} - -function overviewOf(source: string): string { - return markdownOf(predictionRuns().prepared(source).overview('weekly-orders')); -} - -function withDataPart(data: string): string { - const from = weeklyOrdersLines.indexOf('data:'); - const to = weeklyOrdersLines.indexOf('evaluation:'); - return [...weeklyOrdersLines.slice(0, from), data.trimEnd(), ...weeklyOrdersLines.slice(to)].join('\n'); -} - -describe('the overview of the examples of the reference of prediction functions', () => { - it('is, byte for byte, the overview pinned beside this test, for the example and over its uploaded dataset', () => { - expect(overviewOf(example)).toBe(goldenOf('reference-example.overview.md')); - expect(overviewOf(withDataPart(dataPart))).toBe(goldenOf('reference-uploaded.overview.md')); - }); -}); - -describe('the overview of a prediction function of a total to the end of a period', () => { - const overview = overviewOf( - dailyTotalsWith( - [ - ' target: { of: /amount, as: sum }', - ' target: { of: /amount, as: sum }\n features:\n - { name: promo, of: /promo, as: last }\n given: [promo]', - ], - [' activate: automatic', ' shadow_periods: 2\n activate: automatic'], - ), - ); - - it('says the total it predicts, its season, and the given value a question names for the periods it sums', () => { - expect(sectionIn(overview, 'What it predicts')).toBe( - [ - 'It predicts the total of **sales**, counted in sales, for each series from the day after the last one closed to the end of the week that day falls in.', - 'A season is 7 days long.', - 'A question names a series, with the value of **promo** planned for the days it sums.', - 'It answers the last 14 observed days and the total from the next day to the end of its week, with a likely value, a low and a high.', - 'The likely value is the 50th percentile of its model, the low the 10th and the high the 90th.', - ].join('\n\n'), - ); - }); - - it('dates each event by its own time, sums its target, and runs a new version in a shadow of whole periods summed', () => { - expect(sectionIn(overview, 'What it learns from')).toContain( - '- the day, from the time of each event\n- the series, from `/store`\n- **sales**, the sum of `/amount`\n- **promo**, the last value of `/promo`, which a question gives for the days it sums', - ); - expect(sectionIn(overview, 'How a version goes live')).toContain( - 'A version after the first runs beside the active one for 2 weeks, and takes its place only when it predicts no worse.', - ); - }); -}); - -const sentThrough = dailyVisitsWith( - [' horizons: [1, 2]', ' horizons: [1]\n history: 1'], - [ - ' - type: visit.made', - " - { type: visit.made, source: shop, data: '${ $data.paid }' }\n - { type: visit.counted, data: { kind: walk-in } }", - ], - [ - ' activate: automatic', - ' metric: quantile\n baselines: [persistence]\n coverage_floor: 0.75\n confidence: 0.95\n shadow_periods: 1\n activate: { approved: approve-version }', - ], - [' expires: P1D', ' expires: PT12H\n through: send-build'], - [ - 'Fit a model of the daily {{ function.target }} of each of {{ dataset.series }} stores.', - '{% comment %}Kept for the authors of the function.{% endcomment %}', - ], -); - -describe('the overview of a prediction function sent to its trainer and approved through interaction functions', () => { - const overview = overviewOf(sentThrough); - - it('predicts one period ahead, answers the history it names, and names the attributes and the data test of each filter', () => { - expect(sectionIn(overview, 'What it predicts').split('\n\n').slice(0, 3)).toEqual([ - 'It predicts **visits**, counted in visits, for each series a day at a time, 1 day ahead.', - 'A question names a series, and one horizon when it asks for that one alone.', - 'It answers the last observed day and 1 day ahead, with a likely value, a low and a high.', - ]); - expect(sectionIn(overview, 'What it learns from')).toContain( - [ - '- each time an event `visit.made` is published to the playbook, from the source `shop` and when `$data.paid` holds for its data', - '- each time an event `visit.counted` is published to the playbook, when its data is exactly `{"kind":"walk-in"}`', - ].join('\n'), - ); - }); - - it('names the function a build goes through, its wait, and that its brief holds no text a trainer reads', () => { - expect(sectionIn(overview, 'Who trains it')).toBe( - [ - 'Each build asks for candidate models through the interaction function **send-build**.', - 'A build waits at most 12 hours for its candidates, and the playbook then scores up to 8 of them, each within 10 minutes.', - 'Its brief holds no text a trainer reads, so its trainer reads the rows and the rules alone.', - ].join('\n\n'), - ); - }); - - it('says how candidates are scored and that a version goes live once it is approved, and states no rules', () => { - expect(sectionIn(overview, 'How a version goes live')).toBe( - [ - 'The playbook holds back the last 8 days of the window from the trainer, and scores each candidate on them by the quantile loss of its percentiles.', - 'A candidate passes when its interval holds at least 75% of the truths and it beats **persistence** (the last value carried forward) with 95% confidence; the best that passes becomes the next version.', - 'A version after the first runs beside the active one for 1 day, and takes its place only when it predicts no worse.', - 'A version that passes becomes the active one once it is approved through the interaction function **approve-version**.\n', - ].join('\n\n'), - ); - expect(overview).not.toContain('## Rules'); - }); -}); - -describe('the overview of a prediction function over an uploaded dataset of its target alone', () => { - it('names the dataset and its target, and runs no shadow, since the dataset gains no rows', () => { - const overview = overviewOf( - uploadedDailyVisits(sha256).replace( - ' activate: automatic', - ' shadow_periods: 3\n metric: winkler\n activate: automatic', - ), - ); - - expect(sectionIn(overview, 'What it learns from')).toBe( - [ - `It learns from the dataset uploaded to the playbook as \`${sha256}\`, one row a series and a day, with **visits** alone.`, - 'A build reads the last 24 days.', - ].join('\n\n'), - ); - expect(sectionIn(overview, 'How a version goes live')).not.toContain('beside the active one'); - expect(sectionIn(overview, 'How a version goes live')).toContain('by the Winkler score of its interval.'); - }); -}); - -describe('the overview of a prediction function whose document has an about', () => { - it('writes the about as its author wrote it, after the description and before what it predicts', () => { - const description = 'description: Predicts the orders of each region one and four weeks ahead'; - const about = [ - 'about: |', - ' The sales team plans the stock of each region with it.', - '', - ' A person approves each version before it answers.', - ]; - - expect( - overviewOf(weeklyOrdersWith([description, [description, ...about].join('\n')])) - .split('\n\n') - .slice(0, 5), - ).toEqual([ - '# weekly-orders', - 'Predicts the orders of each region one and four weeks ahead', - 'The sales team plans the stock of each region with it.', - 'A person approves each version before it answers.', - '## What it predicts', - ]); - }); -}); diff --git a/capabilities/prediction/src/overview/prediction-overview.ts b/capabilities/prediction/src/overview/prediction-overview.ts deleted file mode 100644 index 42b288cf3..000000000 --- a/capabilities/prediction/src/overview/prediction-overview.ts +++ /dev/null @@ -1,33 +0,0 @@ -import { bulletList, paragraph, sectionOf, type OverviewParts, type OverviewSection } from '@on.auto/definitions'; - -import type { PredictionFunctionDefinitionDocument } from '../document/prediction-document.ts'; -import { learnedSection } from './data-words.ts'; -import { goingLiveSection } from './evaluation-words.ts'; -import { predictedSection } from './predicts-words.ts'; -import { trainerSection } from './trainer-words.ts'; - -function rulesSections(rules: readonly string[]): readonly OverviewSection[] { - return rules.length === 0 - ? [] - : [ - sectionOf( - 'Rules', - paragraph('Its trainer follows these rules, and the card of each version states them:'), - bulletList(rules.map((rule) => [rule])), - ), - ]; -} - -export function predictionOverview(document: PredictionFunctionDefinitionDocument): OverviewParts { - return { - body: [], - sections: [ - predictedSection(document.predicts, document.data), - learnedSection(document), - trainerSection(document.source, document.brief), - goingLiveSection(document), - ...rulesSections(document.rules), - ], - appendix: [], - }; -} diff --git a/capabilities/prediction/src/overview/predicts-words.ts b/capabilities/prediction/src/overview/predicts-words.ts deleted file mode 100644 index 9f9414b58..000000000 --- a/capabilities/prediction/src/overview/predicts-words.ts +++ /dev/null @@ -1,84 +0,0 @@ -import { - listedPhrases, - named, - paragraph, - periodNouns, - sectionOf, - seriesPathBlocks, - type OverviewSection, - type Phrases, -} from '@on.auto/definitions'; -import { listed } from '@on.auto/operations'; - -import { figure } from '../capability/figures.ts'; -import type { DataPart, Predicts } from '../document/prediction-document.ts'; -import type { Grain } from '../periods/grains.ts'; - -export function periodsOf(period: Grain, count: number): string { - const { one, other } = periodNouns[period]; - return `${figure(count)} ${count === 1 ? one : other}`; -} - -export function askedPeriods({ period, accumulates }: Predicts): string { - const { other } = periodNouns[period]; - return accumulates === undefined ? `the ${other} ahead` : `the ${other} it sums`; -} - -function predictedWords({ target, period, horizons, accumulates, unit }: Predicts): Phrases { - if (accumulates !== undefined) { - return [ - 'It predicts the total of ', - named(target), - `, counted in ${unit}, for each series from the ${period} after the last one closed to the end of the ${accumulates} that ${period} falls in.`, - ]; - } - const { one, other } = periodNouns[period]; - const ahead = horizons.length === 1 && horizons[0] === 1 ? one : other; - return [ - 'It predicts ', - named(target), - `, counted in ${unit}, for each series a ${period} at a time, ${listed(horizons.map((horizon) => figure(horizon)))} ${ahead} ahead.`, - ]; -} - -function seasonWords({ period, season }: Predicts): readonly Phrases[] { - return season === undefined ? [] : [[`A season is ${periodsOf(period, season)} long.`]]; -} - -function questionWords(predicts: Predicts, { given }: DataPart): Phrases { - const asked = - predicts.accumulates === undefined - ? 'A question names a series, and one horizon when it asks for that one alone' - : 'A question names a series'; - const planned = - given.length === 0 - ? [] - : [ - ', with the value of ', - ...listedPhrases( - given.map((name) => [named(name)]), - 'and', - ), - ` planned for ${askedPeriods(predicts)}`, - ]; - return [asked, ...planned, '.']; -} - -function answerBlocks({ period, history, horizons, accumulates }: Predicts) { - return seriesPathBlocks({ - period, - observed: history, - ahead: accumulates === undefined ? { periods: horizons } : { totalTo: accumulates }, - }); -} - -export function predictedSection(predicts: Predicts, data: DataPart): OverviewSection { - return sectionOf( - 'What it predicts', - paragraph(...predictedWords(predicts)), - ...seasonWords(predicts).map((words) => paragraph(...words)), - paragraph(...questionWords(predicts, data)), - ...answerBlocks(predicts), - paragraph('The likely value is the 50th percentile of its model, the low the 10th and the high the 90th.'), - ); -} diff --git a/capabilities/prediction/src/overview/reference-example.overview.md b/capabilities/prediction/src/overview/reference-example.overview.md deleted file mode 100644 index 56a2cdea8..000000000 --- a/capabilities/prediction/src/overview/reference-example.overview.md +++ /dev/null @@ -1,62 +0,0 @@ -# weekly-orders - -Predicts the orders of each region one and four weeks ahead - -## What it predicts - -It predicts **orders**, counted in orders, for each series a week at a time, 1 and 4 weeks ahead. - -A season is 52 weeks long. - -A question names a series, and one horizon when it asks for that one alone, with the value of **promotion** planned for the weeks ahead. - -It answers the last 8 observed weeks and 1 and 4 weeks ahead, each with a likely value, a low and a high. - -The likely value is the 50th percentile of its model, the low the 10th and the high the 90th. - -## What it learns from - -It learns from the rows the playbook folds from the first event of its history on: - -- each time an event `order.placed` is published to the playbook - -Each row is one series and one week, and holds: - -- the week, from `/placed_at` in each event's data -- the series, from `/region` -- **orders**, the number of its events -- **basket**, the mean of `/total` -- **promotion**, the last value of `/promotion`, which a question gives for the weeks ahead - -A build reads the last 156 weeks. - -## Who trains it - -Each build asks for candidate models in the playbook's inbox, as a request to **trainer**. - -A build waits at most 7 days for its candidates, and the playbook then scores up to 8 of them, each within 10 minutes. - -Its trainer reads the brief first, with the values of each build in place of the names in bold: - -Fit a model of the weekly **target** of each of **series** regions, in **unit**, -over **window** weeks from **first** to **last**. -Predict **horizons** weeks ahead. The features are -each feature of dataset.features as **name** (**as**), separated by commas. - -*For each rule of function.rules:* - -- **rule** - -## How a version goes live - -The playbook holds back the last 12 weeks of the window from the trainer, and scores each candidate on them by its mean absolute scaled error. - -A candidate passes when its interval holds at least 70% of the truths and it beats **persistence** (the last value carried forward) and **seasonal-naive** (the value one season earlier) with 90% confidence; the best that passes becomes the next version. - -A version that passes becomes the active one on its own. - -## Rules - -Its trainer follows these rules, and the card of each version states them: - -- Train on the rows given and on nothing else diff --git a/capabilities/prediction/src/overview/reference-uploaded.overview.md b/capabilities/prediction/src/overview/reference-uploaded.overview.md deleted file mode 100644 index 2c73e8ed1..000000000 --- a/capabilities/prediction/src/overview/reference-uploaded.overview.md +++ /dev/null @@ -1,55 +0,0 @@ -# weekly-orders - -Predicts the orders of each region one and four weeks ahead - -## What it predicts - -It predicts **orders**, counted in orders, for each series a week at a time, 1 and 4 weeks ahead. - -A season is 52 weeks long. - -A question names a series, and one horizon when it asks for that one alone, with the value of **promotion** planned for the weeks ahead. - -It answers the last 8 observed weeks and 1 and 4 weeks ahead, each with a likely value, a low and a high. - -The likely value is the 50th percentile of its model, the low the 10th and the high the 90th. - -## What it learns from - -It learns from the dataset uploaded to the playbook as `3f2ac1d0e4b5a69788796a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d0e9f8a7b6c5d`, one row a series and a week, with **orders** and then these columns: - -- **basket** -- **promotion**, which a question gives for the weeks ahead - -A build reads the last 156 weeks. - -## Who trains it - -Each build asks for candidate models in the playbook's inbox, as a request to **trainer**. - -A build waits at most 7 days for its candidates, and the playbook then scores up to 8 of them, each within 10 minutes. - -Its trainer reads the brief first, with the values of each build in place of the names in bold: - -Fit a model of the weekly **target** of each of **series** regions, in **unit**, -over **window** weeks from **first** to **last**. -Predict **horizons** weeks ahead. The features are -each feature of dataset.features as **name** (**as**), separated by commas. - -*For each rule of function.rules:* - -- **rule** - -## How a version goes live - -The playbook holds back the last 12 weeks of the window from the trainer, and scores each candidate on them by its mean absolute scaled error. - -A candidate passes when its interval holds at least 70% of the truths and it beats **persistence** (the last value carried forward) and **seasonal-naive** (the value one season earlier) with 90% confidence; the best that passes becomes the next version. - -A version that passes becomes the active one on its own. - -## Rules - -Its trainer follows these rules, and the card of each version states them: - -- Train on the rows given and on nothing else diff --git a/capabilities/prediction/src/overview/trainer-words.ts b/capabilities/prediction/src/overview/trainer-words.ts deleted file mode 100644 index 234343b75..000000000 --- a/capabilities/prediction/src/overview/trainer-words.ts +++ /dev/null @@ -1,31 +0,0 @@ -import { durationInWords, named, paragraph, sectionOf, templateLead, type OverviewSection } from '@on.auto/definitions'; -import { readableTemplate, type TemplateText } from '@on.auto/definitions/template'; - -import { predictionBounds } from '../capability/prediction-bounds.ts'; -import type { BuildSource } from '../document/prediction-document.ts'; - -const { candidatesInABuild, scoringACandidateMs } = predictionBounds; - -function askedThrough({ through, to }: BuildSource) { - return through === undefined - ? paragraph("Each build asks for candidate models in the playbook's inbox, as a request to ", named(to), '.') - : paragraph('Each build asks for candidate models through the interaction function ', named(through), '.'); -} - -function briefBlocks(written: TemplateText) { - const brief = readableTemplate(written); - return brief.length === 0 - ? [paragraph('Its brief holds no text a trainer reads, so its trainer reads the rows and the rules alone.')] - : [templateLead('Its trainer reads the brief first', 'build'), ...brief]; -} - -export function trainerSection(buildSource: BuildSource, brief: TemplateText): OverviewSection { - return sectionOf( - 'Who trains it', - askedThrough(buildSource), - paragraph( - `A build waits at most ${durationInWords(buildSource.expiresMs)} for its candidates, and the playbook then scores up to ${candidatesInABuild} of them, each within ${durationInWords(scoringACandidateMs)}.`, - ), - ...briefBlocks(brief), - ); -} diff --git a/capabilities/reasoning/src/capability/reasoning-function.test.ts b/capabilities/reasoning/src/capability/reasoning-function.test.ts index 68d671546..4e687bddd 100644 --- a/capabilities/reasoning/src/capability/reasoning-function.test.ts +++ b/capabilities/reasoning/src/capability/reasoning-function.test.ts @@ -117,7 +117,7 @@ describe('preparing a reasoning function definition that is not valid', () => { { pointer: '', detail: - 'Line 3, /seed: seed is not a key of the front matter; it takes description, about, model, config, input, output, provider_options, tools', + 'Line 3, /seed: seed is not a key of the front matter; it takes description, model, config, input, output, provider_options, tools', }, ], }), diff --git a/capabilities/reasoning/src/capability/reasoning-function.ts b/capabilities/reasoning/src/capability/reasoning-function.ts index d41b018a6..b7f7b4b75 100644 --- a/capabilities/reasoning/src/capability/reasoning-function.ts +++ b/capabilities/reasoning/src/capability/reasoning-function.ts @@ -9,7 +9,6 @@ import { handCallPresenter } from '@on.auto/tool-servers'; import { parse, summarize } from '../definition/definition-reading.ts'; import type { OfferedModels } from '../model/offered-models.ts'; -import { reasoningOverview } from '../overview/reasoning-overview.ts'; import { longestRequestMs, longestRunMsOf } from './definition-request.ts'; import { definitionRun, type RunServices } from './definition-run.ts'; import { onThisServer } from './on-this-server.ts'; @@ -33,7 +32,6 @@ export function makeReasoningFunctionAdapter(options: ReasoningFunctionAdapterOp mediaType: 'text/markdown', parse, summarize, - overview: reasoningOverview, run: (definition, input, context) => run(definition, input, context), longestAnyRunMs: longestRequestMs, longestRunOf: longestRunMsOf, diff --git a/capabilities/reasoning/src/definition/definition-parsing.test.ts b/capabilities/reasoning/src/definition/definition-parsing.test.ts index 5a2761939..9d31509f9 100644 --- a/capabilities/reasoning/src/definition/definition-parsing.test.ts +++ b/capabilities/reasoning/src/definition/definition-parsing.test.ts @@ -113,7 +113,7 @@ describe('the issues of a document', () => { ).toEqual([ 'Line 2, /model: Expected provider/model, for example anthropic/claude-sonnet-4-5', 'Line 4, /config/temperature: Expected number', - 'Line 5, /flavour: flavour is not a key of the front matter; it takes description, about, model, config, input, output, provider_options, tools', + 'Line 5, /flavour: flavour is not a key of the front matter; it takes description, model, config, input, output, provider_options, tools', 'Line 7: customer is not a variable of a reasoning function’s prompt template, which reads input, today and now; assign it first', ]); }); diff --git a/capabilities/reasoning/src/definition/definition-parsing.ts b/capabilities/reasoning/src/definition/definition-parsing.ts index db38dc0bb..ac716baeb 100644 --- a/capabilities/reasoning/src/definition/definition-parsing.ts +++ b/capabilities/reasoning/src/definition/definition-parsing.ts @@ -56,7 +56,6 @@ function definitionFrom({ root, lines, issues }: ReadFrontMatter, template: () = (section) => outputContractOf(section, lines), ); const description = checkedSection(() => decodeSection.description(root['description']), Result.succeed); - const about = checkedSection(() => decodeSection.about(root['about']), Result.succeed); const providerOptions = checkedSection( () => decodeSection.provider_options(root['provider_options']), (options) => providerOptionsOf(options, lines), @@ -80,7 +79,7 @@ function definitionFrom({ root, lines, issues }: ReadFrontMatter, template: () = ]; return found.length > 0 ? Result.fail(found) - : Result.all({ model, settings, input, output, template: template(), description, about, providerOptions, tools }); + : Result.all({ model, settings, input, output, template: template(), description, providerOptions, tools }); } function definitionOf(parts: DocumentParts): Checked { @@ -93,7 +92,6 @@ function definitionOf(parts: DocumentParts): Checked template), (definition) => ({ ...(definition.description === undefined ? {} : { description: definition.description }), - ...(definition.about === undefined ? {} : { about: definition.about }), model: definition.model, settings: definition.settings, input: definition.input, diff --git a/capabilities/reasoning/src/definition/definition-reading.ts b/capabilities/reasoning/src/definition/definition-reading.ts index af802b9f3..f7ec6e9bf 100644 --- a/capabilities/reasoning/src/definition/definition-reading.ts +++ b/capabilities/reasoning/src/definition/definition-reading.ts @@ -21,14 +21,12 @@ export function parse(source: string): Effect.Effect { ), ), ).toEqual([ - 'Line 3, /prompt: prompt is not a key of the front matter; it takes description, about, model, config, input, output, provider_options, tools', + 'Line 3, /prompt: prompt is not a key of the front matter; it takes description, model, config, input, output, provider_options, tools', 'Line 5, /config/top_k: top_k is not a key of config; it takes max_output_tokens, temperature, top_p, seed, stop_sequences, reasoning', 'Line 7, /input/example: example is not a key of input; it takes schema, default', 'Line 9, /output/strict: strict is not a key of output; it takes format, schema, shape', @@ -70,31 +70,3 @@ describe('the values of the front matter keys', () => { ).toEqual({ openai: { textVerbosity: 'low', reasoningMode: 'pro' } }); }); }); - -describe('the about of the front matter', () => { - it('takes 1 to 4000 characters', () => { - expect(issuesIn(documentOf('model: openai/gpt-5\nabout: ""'))).toEqual([ - 'Line 3, /about: Expected a value with a length of at least 1', - ]); - expect(issuesIn(documentOf(`model: openai/gpt-5\nabout: ${'x'.repeat(4001)}`))).toEqual([ - 'Line 3, /about: Expected a value with a length of at most 4000', - ]); - expect(parsed(documentOf(`model: openai/gpt-5\nabout: ${'x'.repeat(4000)}`)).about).toHaveLength(4000); - }); - - it('takes no heading, nor does the description, each refused at its line', () => { - const frontMatter = [ - 'model: openai/gpt-5', - "description: '## Reviews a brief'", - 'about: |', - ' The campaign team uses it.', - '', - ' # When it runs', - ]; - - expect(issuesIn(documentOf(frontMatter.join('\n')))).toEqual([ - 'Line 3, /description: description takes no heading, since an overview writes its own: the line that begins “## Reviews a brief” makes one, with # before its words or as a line of = or - under the words above it', - 'Line 7, /about: about takes no heading, since an overview writes its own: this line makes one, with # before its words or as a line of = or - under the words above it', - ]); - }); -}); diff --git a/capabilities/reasoning/src/definition/front-matter-schema.ts b/capabilities/reasoning/src/definition/front-matter-schema.ts index 513f06c2a..79cd48a74 100644 --- a/capabilities/reasoning/src/definition/front-matter-schema.ts +++ b/capabilities/reasoning/src/definition/front-matter-schema.ts @@ -1,5 +1,4 @@ import { - AboutSchema, DescriptionSchema, frontMatterIn as sharedFrontMatterIn, type DocumentIssue, @@ -43,7 +42,6 @@ const ProviderOptionsSchema = Schema.Record(Schema.String, Schema.JsonObject); const FrontMatterSchema = Schema.Struct({ description: Schema.optionalKey(DescriptionSchema), - about: Schema.optionalKey(AboutSchema), model: Schema.String, config: Schema.optionalKey(ConfigSchema), input: Schema.optionalKey(InputSectionSchema), @@ -60,7 +58,6 @@ export type OutputSection = typeof OutputSectionSchema.Type | undefined; export const decodeSection = { description: Schema.decodeUnknownOption(Schema.UndefinedOr(DescriptionSchema)), - about: Schema.decodeUnknownOption(Schema.UndefinedOr(AboutSchema)), model: Schema.decodeUnknownOption(Schema.String), config: Schema.decodeUnknownOption(Schema.UndefinedOr(ConfigSchema)), input: Schema.decodeUnknownOption(Schema.UndefinedOr(InputSectionSchema)), diff --git a/capabilities/reasoning/src/definition/reasoning-function-definition.ts b/capabilities/reasoning/src/definition/reasoning-function-definition.ts index 5be803ce6..8f813e173 100644 --- a/capabilities/reasoning/src/definition/reasoning-function-definition.ts +++ b/capabilities/reasoning/src/definition/reasoning-function-definition.ts @@ -12,7 +12,6 @@ export interface InputContract { export interface ReasoningFunctionDefinitionDocument { readonly description?: string; - readonly about?: string; readonly model: string; readonly settings: GenerationSettings; readonly input: InputContract; diff --git a/capabilities/reasoning/src/overview/reasoning-overview.test.ts b/capabilities/reasoning/src/overview/reasoning-overview.test.ts deleted file mode 100644 index 21762c10b..000000000 --- a/capabilities/reasoning/src/overview/reasoning-overview.test.ts +++ /dev/null @@ -1,209 +0,0 @@ -import { readFileSync } from 'node:fs'; - -import { markdownOf, overviewAtRead, type DefinitionUser } from '@on.auto/definitions'; -import { mostReadCharacters } from '@on.auto/definitions/template'; -import { sectionIn } from '@on.auto/definitions/testing'; -import { describe, expect, it } from 'vitest'; - -import { reasoningWith } from '../testing/reasoning-runs.ts'; - -const page = readFileSync(new URL('../../../../docs/reference/reasoning-format.md', import.meta.url), 'utf8'); - -const [, example = ''] = /```markdown\n([\s\S]*?)```/u.exec(page) ?? []; - -const [, toolsOfTheReference = ''] = /```yaml\n(tools: .+)\n```/u.exec(page) ?? []; - -const exampleWithTools = example.replace(/^model: .+$/mu, (modelLine) => `${modelLine}\n${toolsOfTheReference}`); - -const shapesPage = readFileSync(new URL('../../../../docs/reference/output-shapes.md', import.meta.url), 'utf8'); - -const [, shapedExample = ''] = /```markdown\n([\s\S]*?)```/u.exec(shapesPage) ?? []; - -const { prepared } = reasoningWith(); - -const counted = new Intl.NumberFormat('en'); - -function overviewOf(...lines: readonly string[]): string { - return markdownOf(prepared(lines.join('\n')).overview('review-brief')); -} - -function goldenOf(file: string): string { - return readFileSync(new URL(file, import.meta.url), 'utf8'); -} - -function modelAndToolsOf(...tools: readonly string[]): string { - const overview = overviewOf('---', 'model: openai/gpt-5', `tools: [${tools.join(', ')}]`, '---', 'Find it.'); - return sectionIn(overview, 'Which model and tools'); -} - -describe('the overviews of the example of the reference of reasoning functions', () => { - it('are, byte for byte, the overviews pinned beside this test, without tools and with the tools of the reference', () => { - expect(markdownOf(prepared(example).overview('review-brief'))).toBe(goldenOf('reference-example.overview.md')); - expect(markdownOf(prepared(exampleWithTools).overview('review-brief'))).toBe( - goldenOf('reference-tools.overview.md'), - ); - }); -}); - -describe('the overview of the reasoning function of the reference of output shapes', () => { - it('is, byte for byte, the overview pinned beside this test', () => { - expect(markdownOf(prepared(shapedExample).overview('queue-hours'))).toBe( - readFileSync(new URL('shaped-example.overview.md', import.meta.url), 'utf8'), - ); - }); -}); - -describe('the reference of reasoning functions', () => { - it('gives the bound of the instructions and the message an overview shows', () => { - expect(page).toContain(`each cut past ${counted.format(mostReadCharacters)} characters`); - }); -}); - -describe('the overview of a reasoning function whose document has an about', () => { - it('writes the about as its author wrote it, after the description and before what the function needs', () => { - const overview = overviewOf( - '---', - 'description: Reviews a campaign brief', - 'model: openai/gpt-5', - 'about: |', - ' The **campaign team** uses it to check each brief before it goes to a client.', - '', - ' - It runs when someone asks for a review.', - ' - It answers with the evidence the brief lacks.', - '---', - 'Review {{ input.brief }}', - ); - - expect(overview.split('\n\n').slice(0, 5)).toEqual([ - '# review-brief', - 'Reviews a campaign brief', - 'The **campaign team** uses it to check each brief before it goes to a client.', - '- It runs when someone asks for a review.\n- It answers with the evidence the brief lacks.', - '## What it needs', - ]); - }); -}); - -describe('the overview of a reasoning function', () => { - it('says the fields of a JSON answer, and the defaults its input takes in place of what a run leaves out', () => { - const overview = overviewOf( - '---', - 'model: openai/gpt-5', - 'input:', - ' schema: {type: object, properties: {brief: {type: string}, tone: {type: string}, max: true}}', - ' default: {tone: warm, max: 3, other: 1}', - 'output:', - ' format: json', - ' schema: {type: object, properties: {score: {type: integer, maximum: 10}}, required: [score]}', - '---', - 'Score {{ input.brief }}', - ); - - expect(sectionIn(overview, 'What it needs')).toContain('| **tone** | no | Text, “warm” when left out. |'); - expect(sectionIn(overview, 'What it needs')).toContain('| **max** | no | Any value. |'); - expect(sectionIn(overview, 'What it produces')).toContain('| **score** | yes | A whole number, at most 10. |'); - }); -}); - -describe('the overview of a reasoning function whose output names a shape', () => { - it('says the shape in what it produces, in place of the fields of its schema', () => { - const overview = overviewOf('---', 'model: openai/gpt-5', 'output: {shape: text}', '---', 'Write it.'); - - expect(sectionIn(overview, 'What it produces')).toBe('It answers a **text**: Markdown of at most 65,536 bytes.'); - }); -}); - -describe('the model and tools of a reasoning function in its overview', () => { - it('say the model and that it calls no tools, with nothing drawn, when its document names none', () => { - const overview = overviewOf('---', 'model: openai/gpt-5', '---', 'Find it.'); - - expect(sectionIn(overview, 'Which model and tools')).toBe( - ['It reasons with the model `openai/gpt-5`.', 'It calls no tools.'].join('\n\n'), - ); - }); - - it('name each tool with its server, and every tool of a server named with a star', () => { - const [model, mayCall, tools] = modelAndToolsOf('graph/search', 'notes/*').split('\n\n'); - - expect([model, mayCall, tools]).toEqual([ - 'It reasons with the model `openai/gpt-5`.', - 'It may call these tools:', - '- the tool **search** of **graph**\n- every tool of **notes**', - ]); - }); - - it('draw the function, the model it reasons with and each tool the model may call, then say how it calls them', () => { - const [, , , diagram, howItCalls] = modelAndToolsOf('graph/search', 'notes/*').split('\n\n'); - - expect(diagram).toBe( - [ - '```mermaid', - 'flowchart LR', - ' n1["review-brief"]', - ' n2["openai/gpt-5"]', - ' n3["graph/search"]', - ' n4["every tool of notes"]', - ' n1 -->|"reasons with"| n2', - ' n2 -->|"may call"| n3', - ' n2 -->|"may call"| n4', - '```', - ].join('\n'), - ); - expect(howItCalls).toBe( - "The model may call these tools before it answers, several in one step and over several steps, within a run's limits on calls, time and failures.", - ); - }); -}); - -describe('the overview a read writes of a reasoning function whose prompt holds what an overview reads as its own', () => { - const fence = '`'.repeat(3); - const lead = 'Its message, with the values of each run in place of the names in bold:'; - const users: readonly DefinitionUser[] = [ - { type: 'workflow', name: 'weekly-review', relation: 'calls', steps: [{ number: 2, name: 'review' }] }, - ]; - - function partsOf(...prompt: readonly string[]): readonly unknown[] { - const overview = prepared(['---', 'model: openai/gpt-5', '---', ...prompt].join('\n')).overview('review-brief'); - const [story = '', ...details] = overviewAtRead(overview, { users, called: [] }).split('\n## Details\n'); - return [sectionIn(story, 'What it is told'), sectionIn(story, 'Used by'), details]; - } - - const usedBy = '- The workflow **Weekly review** calls it in step 2, **Review**.\n'; - - const detailsOfUsedBy = ['\n### Used by\n\n- `weekly-review`, a workflow\n']; - - it('writes a line ## Details of the prompt with its first character escaped, so the page splits it at its own alone', () => { - expect(partsOf('Review the brief.', '## Details', 'Name what it lacks.')).toEqual([ - `${lead}\n\nReview the brief.\n\\## Details\nName what it lacks.`, - usedBy, - detailsOfUsedBy, - ]); - }); - - it('closes a fence the prompt leaves open at its end, so what follows is read as the overview', () => { - expect(partsOf('Review the brief as in:', fence, 'an example')).toEqual([ - `${lead}\n\nReview the brief as in:\n${fence}\nan example\n${fence}`, - usedBy, - detailsOfUsedBy, - ]); - }); -}); - -describe('the overview of a reasoning function without an input schema or instructions', () => { - it('says what it takes and what it is told', () => { - const overview = overviewOf( - '---', - 'model: openai/gpt-5', - '---', - '{% if input.vip %}Treat them as a VIP.{% endif %} Hello {{ input.name | default: "friend" }}.', - ); - - expect(sectionIn(overview, 'What it needs')).toBe('It takes any input, which its document does not describe.'); - expect(sectionIn(overview, 'What it is told')).toBe( - [ - 'Its message, with the values of each run in place of the names in bold:', - 'Treat them as a VIP when input.vip. Hello **name**.\n', - ].join('\n\n'), - ); - }); -}); diff --git a/capabilities/reasoning/src/overview/reasoning-overview.ts b/capabilities/reasoning/src/overview/reasoning-overview.ts deleted file mode 100644 index f8bca3e6e..000000000 --- a/capabilities/reasoning/src/overview/reasoning-overview.ts +++ /dev/null @@ -1,120 +0,0 @@ -import { - bulletList, - inputSection, - isJsonSchema, - named, - outputSection, - paragraph, - sectionOf, - templateLead, - theToolOf, - type Diagram, - type OverviewBlock, - type OverviewParts, - type OverviewSection, - type Phrases, -} from '@on.auto/definitions'; -import { readableParts } from '@on.auto/definitions/template'; -import { namesEveryTool, writtenOf, type ToolReference } from '@on.auto/tool-servers/policy'; -import type { Schema } from 'effect'; - -import type { - InputContract, - ReasoningFunctionDefinitionDocument, -} from '../definition/reasoning-function-definition.ts'; -import type { OutputRequest } from '../model/model-request.ts'; - -type Entry = readonly [string, Schema.Json]; - -const everyToolOf = 'every tool of '; - -const howTheModelCalls = - "The model may call these tools before it answers, several in one step and over several steps, within a run's limits on calls, time and failures."; - -const functionNode = 0; - -const modelNode = 1; - -const firstToolNode = 2; - -function withDefault([name, property]: Entry, defaults: Schema.JsonObject): Entry { - const value = defaults[name]; - return value === undefined || !isJsonSchema(property) - ? [name, property] - : [name, Object.fromEntries([...Object.entries(property), ['default', value]])]; -} - -function documentedInput({ schema, defaults }: InputContract): Schema.JsonObject | undefined { - const properties = schema?.document['properties']; - const namesItsFields = isJsonSchema(properties); - if (schema === undefined || !namesItsFields) { - return schema?.document; - } - const withDefaults = Object.entries(properties).map((entry: Entry) => withDefault(entry, defaults)); - return Object.fromEntries([...Object.entries(schema.document), ['properties', Object.fromEntries(withDefaults)]]); -} - -function producedSection(output: OutputRequest): OverviewSection { - return output.type === 'json' - ? outputSection(output.schema.document, output.shape) - : sectionOf('What it produces', paragraph('Text: the answer the model writes.')); -} - -function toolPhrases(reference: ToolReference): Phrases { - return namesEveryTool(reference) ? [everyToolOf, named(reference.server)] : theToolOf(reference); -} - -function toolLabel(reference: ToolReference): string { - return namesEveryTool(reference) ? `${everyToolOf}${reference.server}` : writtenOf(reference); -} - -function modelAndTools(name: string, { model, tools }: ReasoningFunctionDefinitionDocument): Diagram { - return { - kind: 'diagram', - nodes: [{ label: name }, { label: model }, ...tools.map((tool) => ({ label: toolLabel(tool) }))], - edges: [ - { from: functionNode, to: modelNode, label: 'reasons with' }, - ...tools.map((_, index) => ({ from: modelNode, to: firstToolNode + index, label: 'may call' })), - ], - }; -} - -function toolBlocks(name: string, definition: ReasoningFunctionDefinitionDocument): readonly OverviewBlock[] { - const { tools } = definition; - return tools.length === 0 - ? [paragraph('It calls no tools.')] - : [ - paragraph('It may call these tools:'), - bulletList(tools.map((tool) => toolPhrases(tool))), - modelAndTools(name, definition), - paragraph(howTheModelCalls), - ]; -} - -function modelSection(name: string, definition: ReasoningFunctionDefinitionDocument): OverviewSection { - return sectionOf( - 'Which model and tools', - paragraph('It reasons with the model ', { code: definition.model }, '.'), - ...toolBlocks(name, definition), - ); -} - -function toldSection({ template }: ReasoningFunctionDefinitionDocument): OverviewSection { - const { inside, outside } = readableParts(template, 'system'); - const instructions = - inside.length === 0 ? [] : [paragraph('Its instructions, which it follows on every run:'), ...inside]; - return sectionOf('What it is told', ...instructions, templateLead('Its message', 'run'), ...outside); -} - -export function reasoningOverview(definition: ReasoningFunctionDefinitionDocument, name: string): OverviewParts { - return { - body: [], - sections: [ - inputSection(documentedInput(definition.input)), - producedSection(definition.output), - modelSection(name, definition), - toldSection(definition), - ], - appendix: [], - }; -} diff --git a/capabilities/reasoning/src/overview/reference-example.overview.md b/capabilities/reasoning/src/overview/reference-example.overview.md deleted file mode 100644 index faee19d60..000000000 --- a/capabilities/reasoning/src/overview/reference-example.overview.md +++ /dev/null @@ -1,31 +0,0 @@ -# review-brief - -Reviews a campaign brief against the team's criteria - -## What it needs - -| Field | Required | What it is | -| --- | --- | --- | -| **brief** | yes | Text. | -| **criteria** | yes | Text. | - -## What it produces - -Text: the answer the model writes. - -## Which model and tools - -It reasons with the model `anthropic/claude-sonnet-4-5`. - -It calls no tools. - -## What it is told - -Its instructions, which it follows on every run: - -Review the brief using the supplied criteria. Identify missing evidence. Do not invent facts. - -Its message, with the values of each run in place of the names in bold: - -Brief: **brief** -Criteria: **criteria** diff --git a/capabilities/reasoning/src/overview/reference-tools.overview.md b/capabilities/reasoning/src/overview/reference-tools.overview.md deleted file mode 100644 index 281a8d6bb..000000000 --- a/capabilities/reasoning/src/overview/reference-tools.overview.md +++ /dev/null @@ -1,50 +0,0 @@ -# review-brief - -Reviews a campaign brief against the team's criteria - -## What it needs - -| Field | Required | What it is | -| --- | --- | --- | -| **brief** | yes | Text. | -| **criteria** | yes | Text. | - -## What it produces - -Text: the answer the model writes. - -## Which model and tools - -It reasons with the model `anthropic/claude-sonnet-4-5`. - -It may call these tools: - -- the tool **search** of **graph** -- the tool **query** of **graph** -- every tool of **notes** - -```mermaid -flowchart LR - n1["review-brief"] - n2["anthropic/claude-sonnet-4-5"] - n3["graph/search"] - n4["graph/query"] - n5["every tool of notes"] - n1 -->|"reasons with"| n2 - n2 -->|"may call"| n3 - n2 -->|"may call"| n4 - n2 -->|"may call"| n5 -``` - -The model may call these tools before it answers, several in one step and over several steps, within a run's limits on calls, time and failures. - -## What it is told - -Its instructions, which it follows on every run: - -Review the brief using the supplied criteria. Identify missing evidence. Do not invent facts. - -Its message, with the values of each run in place of the names in bold: - -Brief: **brief** -Criteria: **criteria** diff --git a/capabilities/reasoning/src/overview/shaped-example.overview.md b/capabilities/reasoning/src/overview/shaped-example.overview.md deleted file mode 100644 index a35b12ae6..000000000 --- a/capabilities/reasoning/src/overview/shaped-example.overview.md +++ /dev/null @@ -1,31 +0,0 @@ -# queue-hours - -Estimates the hours each support queue needs next week, with a range for each - -## What it needs - -| Field | Required | What it is | -| --- | --- | --- | -| **tickets** | yes | The open tickets, one a line, each with its queue. Text. | - -## What it produces - -It answers 1 to 200 **values**, in **unit**, each a **value** under a **name** of its own, with a **low** and a **high** where given. - -## Which model and tools - -It reasons with the model `anthropic/claude-sonnet-4-5`. - -It calls no tools. - -## What it is told - -Its instructions, which it follows on every run: - -Estimate from the tickets alone. Name each queue as the tickets name it. - -Its message, with the values of each run in place of the names in bold: - -For each queue, estimate the hours its team needs next week, with a low and a high, in the unit hours. - -**tickets** diff --git a/capabilities/reasoning/src/template/compiled-template.ts b/capabilities/reasoning/src/template/compiled-template.ts index 59bc809d9..2bdcaf927 100644 --- a/capabilities/reasoning/src/template/compiled-template.ts +++ b/capabilities/reasoning/src/template/compiled-template.ts @@ -1,7 +1,6 @@ import type { RenderClock, RenderFailure as EngineRenderFailure, - TemplateText, VariableReference, } from '@on.auto/definitions/template'; import type { Result, Schema } from 'effect'; @@ -25,7 +24,7 @@ export type RenderFailure = | EngineRenderFailure | { readonly reason: 'too_long'; readonly part: PromptPart; readonly line: number }; -export interface CompiledTemplate extends TemplateText { +export interface CompiledTemplate { readonly hasInstructions: boolean; readonly hasMessage: boolean; readonly variables: readonly VariableReference[]; diff --git a/capabilities/reasoning/src/template/template-compilation.test.ts b/capabilities/reasoning/src/template/template-compilation.test.ts index dae60185d..c4187b677 100644 --- a/capabilities/reasoning/src/template/template-compilation.test.ts +++ b/capabilities/reasoning/src/template/template-compilation.test.ts @@ -21,10 +21,6 @@ describe('a template', () => { }); }); - it('keeps the prompt it was compiled from and the line of the document it starts on, as the overview reads it', () => { - expect(compiled('Hello {{ input.name }}', 6)).toMatchObject({ text: 'Hello {{ input.name }}', firstLine: 6 }); - }); - it('knows when it writes nothing outside the system block', () => { expect(compiled('\n \n').hasMessage).toBe(false); expect(compiled(' {% system %}Be brief. {{ input.x }}{% endsystem %}\n').hasMessage).toBe(false); diff --git a/capabilities/reasoning/src/template/template-compilation.ts b/capabilities/reasoning/src/template/template-compilation.ts index 0492bfd93..a4d6dc6ae 100644 --- a/capabilities/reasoning/src/template/template-compilation.ts +++ b/capabilities/reasoning/src/template/template-compilation.ts @@ -22,8 +22,6 @@ export function compileTemplate( } const hasInstructions = outline.top.length > 0; return Result.succeed({ - text: body, - firstLine, hasInstructions, hasMessage: outline.hasMessage, variables: parsed.variables, diff --git a/capabilities/recall/src/capability/recall-definitions.ts b/capabilities/recall/src/capability/recall-definitions.ts index 5452705c9..e9697a05e 100644 --- a/capabilities/recall/src/capability/recall-definitions.ts +++ b/capabilities/recall/src/capability/recall-definitions.ts @@ -18,14 +18,12 @@ export function parse(source: string): Effect.Effect run({ document, program: stripped.module }, input, context), longestAnyRunMs: deadlineMs, mostActive: mostFunctions, diff --git a/capabilities/recall/src/document/document-parsing.test.ts b/capabilities/recall/src/document/document-parsing.test.ts index a95616f01..9bb5e2296 100644 --- a/capabilities/recall/src/document/document-parsing.test.ts +++ b/capabilities/recall/src/document/document-parsing.test.ts @@ -78,16 +78,16 @@ describe('the front matter of a recall function definition', () => { const front = `${succeeded}\nmodel: anthropic/claude-sonnet-4-5\nconfig: {temperature: 0}\ntools: [graph/*]\ninput: {default: {}}\noutput: {format: json}\nview: {limit: 3}\nsource2: x`; expect(issuesIn(recallDocument(adding, front))).toEqual([ - 'Line 6, /model: model is not a key of the front matter; it takes description, about, language, source, view, input, output', - 'Line 7, /config: config is not a key of the front matter; it takes description, about, language, source, view, input, output', - 'Line 8, /tools: tools is not a key of the front matter; it takes description, about, language, source, view, input, output', + 'Line 6, /model: model is not a key of the front matter; it takes description, language, source, view, input, output', + 'Line 7, /config: config is not a key of the front matter; it takes description, language, source, view, input, output', + 'Line 8, /tools: tools is not a key of the front matter; it takes description, language, source, view, input, output', 'Line 9, /input/default: default is not a key of input; it takes schema', 'Line 10, /output/format: format is not a key of output; it takes schema', 'Line 11, /view/limit: limit is not a key of view; it takes initial, schema', - 'Line 12, /source2: source2 is not a key of the front matter; it takes description, about, language, source, view, input, output', + 'Line 12, /source2: source2 is not a key of the front matter; it takes description, language, source, view, input, output', ]); expect(issuesIn(recallDocument(adding, `${succeeded}\nanswer: '.'`))).toEqual([ - 'Line 6, /answer: answer is not a key of the front matter; it takes description, about, language, source, view, input, output', + 'Line 6, /answer: answer is not a key of the front matter; it takes description, language, source, view, input, output', ]); }); @@ -150,20 +150,3 @@ describe('the module of a recall function definition', () => { ).toBe('export function fold(view: View, event: Event): View {\n return view.perod;\n}'); }); }); - -describe('the about of a recall function definition', () => { - it('is kept as its author wrote it', () => { - expect(parsed(recallDocument(adding, `${succeeded}\nabout: |\n The campaign team asks it.`)).about).toBe( - 'The campaign team asks it.\n', - ); - }); - - it('takes no heading, nor does the description, each refused at its line', () => { - const written = `${succeeded}\ndescription: '# Reviews'\nabout: |\n The campaign team asks it.\n\n ## When`; - - expect(issuesIn(recallDocument(adding, written))).toEqual([ - 'Line 6, /description: description takes no heading, since an overview writes its own: the line that begins “# Reviews” makes one, with # before its words or as a line of = or - under the words above it', - 'Line 10, /about: about takes no heading, since an overview writes its own: this line makes one, with # before its words or as a line of = or - under the words above it', - ]); - }); -}); diff --git a/capabilities/recall/src/document/document-parsing.ts b/capabilities/recall/src/document/document-parsing.ts index 9b85d3966..b6b63d1d7 100644 --- a/capabilities/recall/src/document/document-parsing.ts +++ b/capabilities/recall/src/document/document-parsing.ts @@ -116,14 +116,13 @@ function filterExpressionsOf(filters: readonly ViewFilter[], lines: SourceLines) } function assembled( - { description, about }: Pick, + { description }: Pick, parts: Parts, lines: SourceLines, ): RecallFunctionDefinitionDocument { const { input, output, view, filters, fold } = parts; return { ...(description === undefined ? {} : { description }), - ...(about === undefined ? {} : { about }), language, input, output, diff --git a/capabilities/recall/src/document/front-matter.ts b/capabilities/recall/src/document/front-matter.ts index ad0b738d7..258a44782 100644 --- a/capabilities/recall/src/document/front-matter.ts +++ b/capabilities/recall/src/document/front-matter.ts @@ -1,9 +1,4 @@ -import { - AboutSchema, - DescriptionSchema, - type FrontMatterSection, - type FrontMatterShape, -} from '@on.auto/definitions/document'; +import { DescriptionSchema, type FrontMatterSection, type FrontMatterShape } from '@on.auto/definitions/document'; import { Schema } from 'effect'; const SectionSchema = Schema.Struct({ schema: Schema.optionalKey(Schema.JsonObject) }); @@ -17,7 +12,6 @@ const ViewSchema = Schema.Struct({ const FrontMatterSchema = Schema.Struct({ description: Schema.optionalKey(DescriptionSchema), - about: Schema.optionalKey(AboutSchema), language: Schema.String, source: SourceSchema, view: Schema.optionalKey(ViewSchema), diff --git a/capabilities/recall/src/document/recall-document.ts b/capabilities/recall/src/document/recall-document.ts index ccf04d2c2..db1657086 100644 --- a/capabilities/recall/src/document/recall-document.ts +++ b/capabilities/recall/src/document/recall-document.ts @@ -9,7 +9,6 @@ export interface FilterExpression { export interface RecallFunctionDefinitionDocument { readonly description?: string; - readonly about?: string; readonly language: 'typescript'; readonly input: ValueContract; readonly output: ValueContract; diff --git a/capabilities/recall/src/overview/recall-overview.test.ts b/capabilities/recall/src/overview/recall-overview.test.ts deleted file mode 100644 index d286f8f67..000000000 --- a/capabilities/recall/src/overview/recall-overview.test.ts +++ /dev/null @@ -1,181 +0,0 @@ -import { readFileSync } from 'node:fs'; - -import { markdownOf } from '@on.auto/definitions'; -import { sectionIn } from '@on.auto/definitions/testing'; -import { describe, expect, it } from 'vitest'; - -import { foldOf, recallDocument } from '../testing/campaign-reviews.ts'; -import { recallWith } from '../testing/recall-runs.ts'; - -const page = readFileSync(new URL('../../../../docs/reference/recall-format.md', import.meta.url), 'utf8'); - -const [reviewsExample = '', announcementsExample = ''] = [...page.matchAll(/```markdown\n([\s\S]*?)```/gu)].map( - ([, body = '']: readonly string[]) => body, -); - -const twoFilters = [ - 'language: typescript', - 'source:', - ' events:', - ' - { type: run_succeeded, subject: reasoning/review-brief }', - ' - { type: run_rejected, subject: reasoning/review-brief }', -]; - -function goldenOf(file: string): string { - return readFileSync(new URL(file, import.meta.url), 'utf8'); -} - -function overviewOf(frontMatter: readonly string[], answer?: string): string { - const source = recallDocument(foldOf('return view;', answer), frontMatter.join('\n')); - return markdownOf(recallWith().prepared(source).overview('reviews')); -} - -function rememberedOf(...filters: readonly string[]): readonly string[] { - const events = filters.map((filter) => ` - ${filter}`); - const remembered = sectionIn( - overviewOf(['language: typescript', 'source:', ' events:', ...events]), - 'What it remembers', - ); - return remembered - .split('\n') - .filter((line) => line.startsWith('- ')) - .map((line) => line.slice(2)); -} - -function initialOf(initial: string): string { - return sectionIn(overviewOf([...twoFilters, 'view:', ` initial: ${initial}`]), 'What it keeps'); -} - -describe('the overviews of the examples of the reference of recall functions', () => { - it('are, byte for byte, the overviews pinned beside this test', () => { - const { prepared } = recallWith(); - - expect(markdownOf(prepared(reviewsExample).overview('campaign-reviews'))).toBe( - goldenOf('reference-example.overview.md'), - ); - expect(markdownOf(prepared(announcementsExample).overview('announcements'))).toBe( - goldenOf('reference-common-case.overview.md'), - ); - }); -}); - -describe('the events a recall function remembers, in its overview', () => { - it('say each filter after each time, in the words of an event filter, with each attribute it names', () => { - expect( - rememberedOf( - '{ type: run_succeeded, subject: reasoning/review-brief, caller: acme-admin }', - '{ type: definition_retired, source: /definitions/workflow/greet }', - "{ type: order_placed, data: '${ $data.revenue > 100 }' }", - ), - ).toEqual([ - 'each time a run of the reasoning function **review-brief** succeeds, when the caller is `acme-admin`', - 'each time the workflow **greet** is retired', - 'each time an event `order_placed` is published to the playbook, when `$data.revenue > 100` holds for its data', - ]); - }); -}); - -describe('how the view of a recall function is kept, in its overview', () => { - it('is a diagram from each filter to the view, drawn as a cylinder, and from the view to the answer', () => { - const overview = overviewOf(twoFilters, 'return view;'); - - expect(sectionIn(overview, 'How it is kept').split('\n\n')[0]).toBe( - [ - '```mermaid', - 'flowchart LR', - ' n1["A run of the reasoning function review-brief succeeds"]', - ' n2["A run of the reasoning function review-brief is rejected"]', - ' n3[("The view")]', - ' n4["The answer of a run"]', - ' n1 --> n3', - ' n2 --> n3', - ' n3 --> n4', - '```', - ].join('\n'), - ); - expect(sectionIn(overview, 'What it remembers')).toMatch( - /\n\nIt remembers an event that matches more than one of these once\.$/u, - ); - }); -}); - -describe('what a recall function keeps and answers, in its overview', () => { - it('says the view it starts with, unless it starts empty', () => { - expect(initialOf('{ count: 0 }')).toMatch(/\n\nBefore the first event, the view is `\{"count":0\}`\.$/u); - expect(initialOf('0')).toMatch(/\n\nBefore the first event, the view is `0`\.$/u); - expect(initialOf(`'${'x'.repeat(300)}'`)).toMatch( - /\n\nBefore the first event, the view is the value of `view\.initial`, shown under Source\.$/u, - ); - expect(['{}', '[]', "''", 'null'].map((initial) => initialOf(initial))).toEqual( - ['{}', '[]', "''", 'null'].map(() => - [ - 'It keeps one view, which its program updates with each event it remembers:', - 'Any value, which its document does not describe.', - ].join('\n\n'), - ), - ); - }); - - it('says a function without an answer answers with its view, and the schema its view is checked against', () => { - const answered = ['Each run takes:', 'Any input, which its document does not describe.']; - - expect(sectionIn(overviewOf(twoFilters), 'What it answers')).toBe( - [...answered, 'It answers with the view itself, as it stands.'].join('\n\n'), - ); - expect(sectionIn(overviewOf([...twoFilters, 'output:', ' schema: { type: object }']), 'What it answers')).toBe( - [...answered, 'It answers with the view itself, as it stands.', 'A record.'].join('\n\n'), - ); - }); - - it('says what a function with an answer but no schemas takes and answers', () => { - expect(sectionIn(overviewOf(twoFilters, 'return view;'), 'What it answers')).toBe( - [ - 'Each run takes:', - 'Any input, which its document does not describe.', - 'It answers with what its program makes of the view and that input:', - 'Any output, which its document does not describe.', - ].join('\n\n'), - ); - }); -}); - -describe('the view a recall function starts with, in its overview', () => { - it('is said within 300 characters and named under Source at 301, a character beyond the Basic Multilingual Plane counted once', () => { - const underSource = 'Before the first event, the view is the value of `view.initial`, shown under Source.'; - - expect( - [`'${'x'.repeat(298)}'`, `'${'x'.repeat(299)}'`, `'${'😀'.repeat(298)}'`, `'${'😀'.repeat(299)}'`].map( - (initial) => initialOf(initial).split('\n\n').at(-1), - ), - ).toEqual([ - `Before the first event, the view is \`"${'x'.repeat(298)}"\`.`, - underSource, - `Before the first event, the view is \`"${'😀'.repeat(298)}"\`.`, - underSource, - ]); - }); -}); - -describe('the overview of a recall function whose document has an about', () => { - it('writes the about as its author wrote it, after the description and before what it remembers', () => { - const about = [ - 'description: The reviews of each campaign', - 'about: |', - ' The campaign team asks it what the reviews of a campaign have said so far.', - '', - ' It answers with them latest last.', - ]; - - expect( - overviewOf([...twoFilters, ...about]) - .split('\n\n') - .slice(0, 5), - ).toEqual([ - '# reviews', - 'The reviews of each campaign', - 'The campaign team asks it what the reviews of a campaign have said so far.', - 'It answers with them latest last.', - '## What it remembers', - ]); - }); -}); diff --git a/capabilities/recall/src/overview/recall-overview.ts b/capabilities/recall/src/overview/recall-overview.ts deleted file mode 100644 index 24e4b06a7..000000000 --- a/capabilities/recall/src/overview/recall-overview.ts +++ /dev/null @@ -1,119 +0,0 @@ -import { - bulletList, - charactersIn, - describedSchemaBlocks, - eventFilterPhrases, - paragraph, - plainTextOf, - schemaBlocks, - sectionOf, - shownUnderSource, - undescribedSentence, - type Diagram, - type OverviewBlock, - type OverviewParts, - type OverviewSection, -} from '@on.auto/definitions'; -import { capitalized } from '@on.auto/operations'; -import type { ViewFilter } from '@on.auto/workflow-host'; -import type { Schema } from 'effect'; - -import type { RecallFunctionDefinitionDocument } from '../document/recall-document.ts'; - -const mostInitialViewCharacters = 300; - -function remembersSection(filters: readonly ViewFilter[]): OverviewSection { - const once = filters.length > 1 ? [paragraph('It remembers an event that matches more than one of these once.')] : []; - return sectionOf( - 'What it remembers', - paragraph("It remembers, from the first event of the playbook's history on:"), - bulletList(filters.map((filter) => ['each time ', ...eventFilterPhrases(filter)])), - ...once, - ); -} - -function isEmpty(value: Schema.Json): boolean { - return value === null || value === '' || (typeof value === 'object' && Object.keys(value).length === 0); -} - -function initialBlocks(initial: Schema.Json): readonly OverviewBlock[] { - if (isEmpty(initial)) { - return []; - } - const written = JSON.stringify(initial); - return charactersIn(written) > mostInitialViewCharacters - ? [shownUnderSource('value', 'Before the first event, the view is the value of ', { code: 'view.initial' })] - : [paragraph('Before the first event, the view is ', { code: written }, '.')]; -} - -function keepsSection({ details }: RecallFunctionDefinitionDocument): OverviewSection { - return sectionOf( - 'What it keeps', - paragraph('It keeps one view, which its program updates with each event it remembers:'), - ...schemaBlocks(details.schema, undescribedSentence('value', 'alone')), - ...initialBlocks(details.initial), - ); -} - -function answerBlocks({ output, answers }: RecallFunctionDefinitionDocument): readonly OverviewBlock[] { - if (answers) { - return [ - paragraph('It answers with what its program makes of the view and that input:'), - ...schemaBlocks(output.schema?.document, undescribedSentence('output', 'alone')), - ]; - } - const checked = output.schema === undefined ? [] : describedSchemaBlocks(output.schema.document); - return [paragraph('It answers with the view itself, as it stands.'), ...checked]; -} - -function answersSection(document: RecallFunctionDefinitionDocument): OverviewSection { - return sectionOf( - 'What it answers', - paragraph('Each run takes:'), - ...schemaBlocks(document.input.schema?.document, undescribedSentence('input', 'alone')), - ...answerBlocks(document), - ); -} - -function diagramOf(filters: readonly ViewFilter[], answers: boolean): Diagram { - const view = filters.length; - return { - kind: 'diagram', - nodes: [ - ...filters.map((filter) => ({ label: capitalized(plainTextOf(eventFilterPhrases(filter))) })), - { label: 'The view', shape: 'cylinder' }, - { label: answers ? 'The answer of a run' : 'The answer of a run, the view itself' }, - ], - edges: [...filters.map((_, index) => ({ from: index, to: view })), { from: view, to: view + 1 }], - }; -} - -function keptSection({ details, answers }: RecallFunctionDefinitionDocument): OverviewSection { - return sectionOf( - 'How it is kept', - diagramOf(details.filters, answers), - paragraph( - "Its view is built from the first event of the playbook's history when the function is saved, and again whenever it changes, and is kept up to date as the events it remembers arrive.", - ), - shownUnderSource( - 'program', - 'The program, its ', - { code: 'fold' }, - ...(answers ? [' and its ', { code: 'answer' }] : []), - ',', - ), - ); -} - -export function recallOverview(document: RecallFunctionDefinitionDocument): OverviewParts { - return { - body: [], - sections: [ - remembersSection(document.details.filters), - keepsSection(document), - answersSection(document), - keptSection(document), - ], - appendix: [], - }; -} diff --git a/capabilities/recall/src/overview/reference-common-case.overview.md b/capabilities/recall/src/overview/reference-common-case.overview.md deleted file mode 100644 index 487e5b61f..000000000 --- a/capabilities/recall/src/overview/reference-common-case.overview.md +++ /dev/null @@ -1,38 +0,0 @@ -# announcements - -What the post-announcement function returned, oldest first, the last 50 - -## What it remembers - -It remembers, from the first event of the playbook's history on: - -- each time a run of the reasoning function **post-announcement** succeeds - -## What it keeps - -It keeps one view, which its program updates with each event it remembers: - -A list of at most 50 items. - -## What it answers - -Each run takes: - -Any input, which its document does not describe. - -It answers with the view itself, as it stands. - -## How it is kept - -```mermaid -flowchart LR - n1["A run of the reasoning function post-announcement succeeds"] - n2[("The view")] - n3["The answer of a run, the view itself"] - n1 --> n2 - n2 --> n3 -``` - -Its view is built from the first event of the playbook's history when the function is saved, and again whenever it changes, and is kept up to date as the events it remembers arrive. - -The program, its `fold`, is TypeScript and is shown under Source. diff --git a/capabilities/recall/src/overview/reference-example.overview.md b/capabilities/recall/src/overview/reference-example.overview.md deleted file mode 100644 index 911519ec7..000000000 --- a/capabilities/recall/src/overview/reference-example.overview.md +++ /dev/null @@ -1,43 +0,0 @@ -# campaign-reviews - -The reviews of each campaign, latest last, as the review-brief function wrote them - -## What it remembers - -It remembers, from the first event of the playbook's history on: - -- each time a run of the reasoning function **review-brief** succeeds - -## What it keeps - -It keeps one view, which its program updates with each event it remembers: - -A record of at most 50 entries, each a list of at most 20 records with: **at** (text), **verdict** (text), **run** (text). - -## What it answers - -Each run takes: - -| Field | Required | What it is | -| --- | --- | --- | -| **campaign** | yes | Text. | -| **last** | no | A whole number, from 1 to 50. | - -It answers with what its program makes of the view and that input: - -A list of records with: **at** (text), **verdict** (text), **run** (text, optional). - -## How it is kept - -```mermaid -flowchart LR - n1["A run of the reasoning function review-brief succeeds"] - n2[("The view")] - n3["The answer of a run"] - n1 --> n2 - n2 --> n3 -``` - -Its view is built from the first event of the playbook's history when the function is saved, and again whenever it changes, and is kept up to date as the events it remembers arrive. - -The program, its `fold` and its `answer`, is TypeScript and is shown under Source. diff --git a/docs/assets/openapi.json b/docs/assets/openapi.json index 31f58e7dd..6b0174878 100644 --- a/docs/assets/openapi.json +++ b/docs/assets/openapi.json @@ -5003,7 +5003,7 @@ "post": { "operationId": "create_definition", "summary": "Create definition", - "description": "Saves a new function or workflow definition in the playbook from its document and returns it without running it. Use it once the person has agreed to the definition; update_definition changes one that exists, and a name is never reused in a playbook. `type` is the definition's type and `name` is how workflows and tools refer to it: reasoning, a reasoning function; interaction, an interaction function; computation, a computation function; recall, a recall function; prediction, a prediction function; workflow, a workflow. `source` is the whole document in that type's format, which get_guide gives. A document that does not fit its format is refused with the line and what is wrong, and nothing is saved.", + "description": "Saves a new function or workflow definition in the playbook from its document and returns it without running it. Use it once the person has agreed to the definition; update_definition changes one that exists, and a name is never reused in a playbook. `type` is the definition's type and `name` is how workflows and tools refer to it: reasoning, a reasoning function; interaction, an interaction function; computation, a computation function; recall, a recall function; prediction, a prediction function; workflow, a workflow. `source` is the whole document in that type's format, which get_guide gives, and `overview` the Markdown written after it for the person who runs the business. A document that does not fit its format is refused with the line and what is wrong, and nothing is saved.", "tags": [ "definitions" ], @@ -5051,6 +5051,12 @@ "type": "string", "description": "The definition document in its type's format, at most 65536 bytes in UTF-8", "maxLength": 65536 + }, + "overview": { + "type": "string", + "description": "Markdown for the person who runs the business, written from everything you know of them and of the playbook: why this exists, what it does and how it fits, in one to three short paragraphs, with a Mermaid diagram in a fenced block where a flow is worth a picture; less technical, more value.", + "minLength": 1, + "maxLength": 16384 } }, "required": [ @@ -5082,7 +5088,7 @@ "version": { "type": "integer", "minimum": 1, - "description": "The definition version: 1 when created, and one more for every update that changes its document" + "description": "The definition version: 1 when created, and one more for every update that changes its document or its overview" }, "status": { "type": "string", @@ -5255,7 +5261,7 @@ }, "overview": { "type": "string", - "description": "The definition in plain words for a person who does not read its document: Markdown with headings, lists, tables of its fields and, where one helps, a diagram, saying, after its description and its about, as their author wrote them, what it needs and produces and what it does, as its type has it, composed by the runtime from the document when this version was saved and written each time it is read, with, after what the document says, the definitions a workflow calls, each with its description, and the active definitions of the playbook that use this one, a workflow with the steps that call it, and their exact names under Details at the end" + "description": "The overview its author wrote: Markdown for the person who runs the business, kept as written with this version" }, "triggers_since": { "type": "string", @@ -5275,8 +5281,7 @@ "created_at", "created_by", "updated_at", - "source", - "overview" + "source" ], "additionalProperties": false, "description": "A saved definition, with its document" @@ -5406,7 +5411,7 @@ "version": { "type": "integer", "minimum": 1, - "description": "The definition version: 1 when created, and one more for every update that changes its document" + "description": "The definition version: 1 when created, and one more for every update that changes its document or its overview" }, "status": { "type": "string", @@ -5699,7 +5704,7 @@ "version": { "type": "integer", "minimum": 1, - "description": "The definition version: 1 when created, and one more for every update that changes its document" + "description": "The definition version: 1 when created, and one more for every update that changes its document or its overview" }, "status": { "type": "string", @@ -5872,7 +5877,7 @@ }, "overview": { "type": "string", - "description": "The definition in plain words for a person who does not read its document: Markdown with headings, lists, tables of its fields and, where one helps, a diagram, saying, after its description and its about, as their author wrote them, what it needs and produces and what it does, as its type has it, composed by the runtime from the document when this version was saved and written each time it is read, with, after what the document says, the definitions a workflow calls, each with its description, and the active definitions of the playbook that use this one, a workflow with the steps that call it, and their exact names under Details at the end" + "description": "The overview its author wrote: Markdown for the person who runs the business, kept as written with this version" }, "triggers_since": { "type": "string", @@ -5892,8 +5897,7 @@ "created_at", "created_by", "updated_at", - "source", - "overview" + "source" ], "additionalProperties": false, "description": "A saved definition, with its document" @@ -5946,7 +5950,7 @@ "put": { "operationId": "update_definition", "summary": "Update definition", - "description": "Replaces the whole document of an active function or workflow definition and returns its new version, which every run uses from then on. Use it when the person changes a saved definition; create_definition saves a new one, and a retired definition cannot change. `type` and `name` say which definition, and `source` is the whole new document in its type's format, which get_guide gives. A document that does not fit its format is refused with the line and what is wrong, and one the same as the saved document records nothing. A new version of a recall function builds its view again from the playbook's history.", + "description": "Replaces the whole document of an active function or workflow definition and returns its new version, which every run uses from then on. Use it when the person changes a saved definition; create_definition saves a new one, and a retired definition cannot change. `type` and `name` say which definition, `source` is the whole new document in its type's format, which get_guide gives, and `overview` the Markdown for the person who runs the business, written again with every update; left out, the saved overview stays. A document that does not fit its format is refused with the line and what is wrong, and one the same as the saved document, with its overview the same or left out, records nothing. A new version of a recall function builds its view again from the playbook's history.", "tags": [ "definitions" ], @@ -5999,6 +6003,12 @@ "type": "string", "description": "The definition document in its type's format, at most 65536 bytes in UTF-8", "maxLength": 65536 + }, + "overview": { + "type": "string", + "description": "Markdown for the person who runs the business, written from everything you know of them and of the playbook: why this exists, what it does and how it fits, in one to three short paragraphs, with a Mermaid diagram in a fenced block where a flow is worth a picture; less technical, more value.", + "minLength": 1, + "maxLength": 16384 } }, "required": [ @@ -6029,7 +6039,7 @@ "version": { "type": "integer", "minimum": 1, - "description": "The definition version: 1 when created, and one more for every update that changes its document" + "description": "The definition version: 1 when created, and one more for every update that changes its document or its overview" }, "status": { "type": "string", @@ -6202,7 +6212,7 @@ }, "overview": { "type": "string", - "description": "The definition in plain words for a person who does not read its document: Markdown with headings, lists, tables of its fields and, where one helps, a diagram, saying, after its description and its about, as their author wrote them, what it needs and produces and what it does, as its type has it, composed by the runtime from the document when this version was saved and written each time it is read, with, after what the document says, the definitions a workflow calls, each with its description, and the active definitions of the playbook that use this one, a workflow with the steps that call it, and their exact names under Details at the end" + "description": "The overview its author wrote: Markdown for the person who runs the business, kept as written with this version" }, "triggers_since": { "type": "string", @@ -6222,8 +6232,7 @@ "created_at", "created_by", "updated_at", - "source", - "overview" + "source" ], "additionalProperties": false, "description": "A saved definition, with its document" @@ -6355,7 +6364,7 @@ "version": { "type": "integer", "minimum": 1, - "description": "The definition version: 1 when created, and one more for every update that changes its document" + "description": "The definition version: 1 when created, and one more for every update that changes its document or its overview" }, "status": { "type": "string", @@ -6528,7 +6537,7 @@ }, "overview": { "type": "string", - "description": "The definition in plain words for a person who does not read its document: Markdown with headings, lists, tables of its fields and, where one helps, a diagram, saying, after its description and its about, as their author wrote them, what it needs and produces and what it does, as its type has it, composed by the runtime from the document when this version was saved and written each time it is read, with, after what the document says, the definitions a workflow calls, each with its description, and the active definitions of the playbook that use this one, a workflow with the steps that call it, and their exact names under Details at the end" + "description": "The overview its author wrote: Markdown for the person who runs the business, kept as written with this version" }, "triggers_since": { "type": "string", @@ -6548,8 +6557,7 @@ "created_at", "created_by", "updated_at", - "source", - "overview" + "source" ], "additionalProperties": false, "description": "A saved definition, with its document" diff --git a/packages/api/src/mcp/instructions.test.ts b/packages/api/src/mcp/instructions.test.ts index 2e9af5d04..8f1a94a19 100644 --- a/packages/api/src/mcp/instructions.test.ts +++ b/packages/api/src/mcp/instructions.test.ts @@ -15,13 +15,13 @@ import { import { instructionsFor, type ServedTools } from './instructions.ts'; const servedOnMcp = - "Auto Runtime executes playbooks; Studio manages them; your agent is a separate client. A playbook contains one brain (its reusable functions) and workflows. The brain shares its lifecycle. Functions run directly or in workflows; runs belong to the playbook. A reasoning function has a prompt and calls a language model. An interaction function asks a system and answers at once, or a person and answers started. A computation function runs a program whose output is the same for the same input. Recall retrieves run results, definitions and events, never run inputs or tool calls. What the person wants kept goes in a recall function, for anyone or a workflow to ask later, not your notes. A prediction function answers from a model trained on the playbook's history; a build, input { build: {} }, answers started. A workflow runs functions in steps, waits for input and can start on a schedule or an event. This connection acts in the caller's own org: list_playbooks shows its playbooks and create_playbook makes one. Before writing a definition, read its format with get_guide. A reasoning function names a model that list_models lists and may name tools that search finds. Accounts at services are connected through this server: asked to connect, link, sign in to or use a service, call search with its words, which lists the providers that match, then add_connection with a provider's name, which the person approves on the page it links. Say a service cannot be connected only when no provider or server matches. A run of a workflow answers started; get_run shows whether it ended or still waits. When the person approves, rejects or otherwise answers what a run waits on, answer its request with answer_interaction, not a new run. Say what was done and what the person can do next, in the words of playbooks, functions, workflows and runs, not the tools' names, fields or rules; an id or status only where needed to act. A tool that cannot do what was asked says why and what to change."; + "Auto Runtime executes playbooks; Studio manages them; your agent is a separate client. A playbook contains one brain (its reusable functions) and workflows. The brain shares its lifecycle. Functions run directly or in workflows; runs belong to the playbook. A reasoning function has a prompt and calls a language model. An interaction function asks a system and answers at once, or a person and answers started. A computation function runs a program whose output is the same for the same input. Recall retrieves run results, definitions and events, never run inputs or tool calls. What the person wants kept goes in a recall function, for anyone or a workflow to ask later, not your notes. A prediction function answers from a model trained on the playbook's history; a build, input { build: {} }, answers started. A workflow runs functions in steps, waits for input and can start on a schedule or an event. This connection acts in the caller's own org: list_playbooks shows its playbooks and create_playbook makes one. Read a definition's format with get_guide; give it an overview. A reasoning function names a model that list_models lists and may name tools that search finds. Accounts at services are connected through this server: asked to connect, link, sign in to or use a service, call search with its words, which lists the providers that match, then add_connection with a provider's name, which the person approves on the page it links. Say a service cannot be connected only when no provider or server matches. A run of a workflow answers started; get_run shows whether it ended or still waits. When the person approves, rejects or otherwise answers what a run waits on, answer its request with answer_interaction, not a new run. Say what was done and what the person can do next, in the words of playbooks, functions, workflows and runs, not the tools' names, fields or rules; an id or status only where needed to act. A tool that cannot do what was asked says why and what to change."; const servedOnAnOrg = "Auto Runtime executes playbooks; Studio manages them; your agent is a separate client. A playbook contains one brain (its reusable functions) and workflows. The brain shares its lifecycle. Functions run directly or in workflows; runs belong to the playbook. A reasoning function has a prompt and calls a language model. An interaction function asks a system and answers at once, or a person and answers started. A computation function runs a program whose output is the same for the same input. Recall retrieves run results, definitions and events, never run inputs or tool calls. What the person wants kept goes in a recall function, for anyone or a workflow to ask later, not your notes. A prediction function answers from a model trained on the playbook's history; a build, input { build: {} }, answers started. A workflow runs functions in steps, waits for input and can start on a schedule or an event. This connection manages the playbooks of one org: list_playbooks shows them and create_playbook makes one, and a playbook's functions and workflows are made on the playbook's own connection, /orgs/{org}/playbooks/{playbook}/mcp. get_guide holds what these words mean and how each kind of definition is written. A reasoning function names a model that list_models lists and may name tools that search finds. Accounts at services are connected through this server: asked to connect, link, sign in to or use a service, call search with its words, which lists the providers that match, then add_connection with a provider's name, which the person approves on the page it links. Say a service cannot be connected only when no provider or server matches. Say what was done and what the person can do next, in the words of playbooks, functions, workflows and runs, not the tools' names, fields or rules; an id or status only where needed to act. A tool that cannot do what was asked says why and what to change."; const servedOnAPlaybook = - "Auto Runtime executes playbooks; Studio manages them; your agent is a separate client. A playbook contains one brain (its reusable functions) and workflows. The brain shares its lifecycle. Functions run directly or in workflows; runs belong to the playbook. A reasoning function has a prompt and calls a language model. An interaction function asks a system and answers at once, or a person and answers started. A computation function runs a program whose output is the same for the same input. Recall retrieves run results, definitions and events, never run inputs or tool calls. What the person wants kept goes in a recall function, for anyone or a workflow to ask later, not your notes. A prediction function answers from a model trained on the playbook's history; a build, input { build: {} }, answers started. A workflow runs functions in steps, waits for input and can start on a schedule or an event. This connection acts inside one playbook. Before writing a definition, read its format with get_guide. A reasoning function names a model this server can call, as the reasoning-function guide says, and may name tools that search finds. A run of a workflow answers started; get_run shows whether it ended or still waits. When the person approves, rejects or otherwise answers what a run waits on, answer its request with answer_interaction, not a new run. Say what was done and what the person can do next, in the words of playbooks, functions, workflows and runs, not the tools' names, fields or rules; an id or status only where needed to act. A tool that cannot do what was asked says why and what to change."; + "Auto Runtime executes playbooks; Studio manages them; your agent is a separate client. A playbook contains one brain (its reusable functions) and workflows. The brain shares its lifecycle. Functions run directly or in workflows; runs belong to the playbook. A reasoning function has a prompt and calls a language model. An interaction function asks a system and answers at once, or a person and answers started. A computation function runs a program whose output is the same for the same input. Recall retrieves run results, definitions and events, never run inputs or tool calls. What the person wants kept goes in a recall function, for anyone or a workflow to ask later, not your notes. A prediction function answers from a model trained on the playbook's history; a build, input { build: {} }, answers started. A workflow runs functions in steps, waits for input and can start on a schedule or an event. This connection acts inside one playbook. Read a definition's format with get_guide; give it an overview. A reasoning function names a model this server can call, as the reasoning-function guide says, and may name tools that search finds. A run of a workflow answers started; get_run shows whether it ended or still waits. When the person approves, rejects or otherwise answers what a run waits on, answer its request with answer_interaction, not a new run. Say what was done and what the person can do next, in the words of playbooks, functions, workflows and runs, not the tools' names, fields or rules; an id or status only where needed to act. A tool that cannot do what was asked says why and what to change."; const interactionSentence = 'An interaction function asks a system and answers at once, or a person and answers started.'; @@ -188,14 +188,14 @@ describe('what the instructions say of the definition types', () => { expect( ['computation function', 'recall function', 'A workflow runs'].filter((words) => instructions.includes(words)), ).toEqual([]); - expect(instructions).toContain('This connection acts inside one playbook. Before writing a definition'); + expect(instructions).toContain("This connection acts inside one playbook. Read a definition's format"); }); it('give no sentence to a type they have no words for', () => { const drafting = [{ type: 'drafting', noun: 'draft', guide: 'drafting' }]; expect(instructionsFor('playbook', playbookEndpoint, drafting)).toContain( - 'Auto Runtime executes playbooks; Studio manages them; your agent is a separate client. A playbook contains one brain (its reusable functions) and workflows. The brain shares its lifecycle. Functions run directly or in workflows; runs belong to the playbook. This connection acts inside one playbook. Before writing a definition', + "Auto Runtime executes playbooks; Studio manages them; your agent is a separate client. A playbook contains one brain (its reusable functions) and workflows. The brain shares its lifecycle. Functions run directly or in workflows; runs belong to the playbook. This connection acts inside one playbook. Read a definition's format", ); }); @@ -214,14 +214,14 @@ describe('what the instructions say of the definition types', () => { }); describe('what the instructions say of the guides', () => { - it("point to get_guide for a definition's format on every endpoint that writes one, and name no recipe, which get_guide names", () => { + it("point to get_guide for a definition's format and ask for its overview on every endpoint that writes one, and name no recipe, which get_guide names", () => { const writing = everyEndpoint.filter(([endpoint]) => endpoint !== 'org'); expect( writing.filter( ([endpoint, served]) => !instructionsFor(endpoint, served, definitionTypes).includes( - 'Before writing a definition, read its format with get_guide. A reasoning function', + "Read a definition's format with get_guide; give it an overview. A reasoning function", ), ), ).toEqual([]); diff --git a/packages/api/src/mcp/instructions.ts b/packages/api/src/mcp/instructions.ts index dfded2309..8106134ca 100644 --- a/packages/api/src/mcp/instructions.ts +++ b/packages/api/src/mcp/instructions.ts @@ -86,7 +86,7 @@ const modelsBeforeWriting: Sentences = ({ reasoning, listed }) => const guides: Sentences = ({ listed }) => [ listed('create_definition') - ? 'Before writing a definition, read its format with get_guide.' + ? "Read a definition's format with get_guide; give it an overview." : 'get_guide holds what these words mean and how each kind of definition is written.', ]; diff --git a/packages/api/src/mcp/tool-calling-instructions.test.ts b/packages/api/src/mcp/tool-calling-instructions.test.ts index 6bb089d8f..79265de86 100644 --- a/packages/api/src/mcp/tool-calling-instructions.test.ts +++ b/packages/api/src/mcp/tool-calling-instructions.test.ts @@ -12,10 +12,10 @@ import { import { instructionsFor } from './instructions.ts'; describe('the length of the instructions with every tool', () => { - it('take 1,993 characters on /mcp, 1,912 on the org endpoint and 1,618 on the playbook endpoint', () => { + it('take 1,996 characters on /mcp, 1,912 on the org endpoint and 1,621 on the playbook endpoint', () => { expect( everyEndpoint.map(([endpoint, served]) => instructionsFor(endpoint, served, definitionTypes).length), - ).toEqual([1912, 1618, 1993]); + ).toEqual([1912, 1621, 1996]); }); }); diff --git a/packages/definitions/measure/measure-loads.ts b/packages/definitions/measure/measure-loads.ts index 0692a9335..b79e822ff 100644 --- a/packages/definitions/measure/measure-loads.ts +++ b/packages/definitions/measure/measure-loads.ts @@ -17,6 +17,8 @@ const samples = Number(process.env['DEFINITIONS_MEASURE_SAMPLES'] ?? 15); const mostSourceBytes = 65_536; +const mostOverviewBytes = 16_384; + const utf8 = new TextEncoder(); const decodeFields = Schema.decodeUnknownEffect(Schema.fromJsonString(Schema.Record(Schema.String, Schema.String))); @@ -64,6 +66,13 @@ function documentOf(version: number): string { return `{${entries.join(',')}}`; } +function overviewOf(version: number): string { + return `Version ${version} of the overview. `.padEnd( + mostOverviewBytes, + 'What the person who runs the business reads. ', + ); +} + function median(values: readonly number[]): number { const sorted = values.toSorted((a, b) => a - b); return sorted[Math.floor(sorted.length / 2)] ?? 0; @@ -138,10 +147,21 @@ const runtime = ManagedRuntime.make( try { const { load, read } = await runtime.runPromise( Effect.gen(function* () { - yield* dispatched(createDefinition, { type: 'widest', name: 'widest', source: documentOf(1) }); + yield* dispatched(createDefinition, { + type: 'widest', + name: 'widest', + source: documentOf(1), + overview: overviewOf(1), + }); yield* Effect.forEach( Array.from({ length: versions - 1 }, (_, index) => index + 2), - (version) => dispatched(updateDefinition, { type: 'widest', name: 'widest', source: documentOf(version) }), + (version) => + dispatched(updateDefinition, { + type: 'widest', + name: 'widest', + source: documentOf(version), + overview: overviewOf(version), + }), ); return { load: yield* sampled(dispatched(listDefinitions, { type: 'widest' })), diff --git a/packages/definitions/src/capability/capability.ts b/packages/definitions/src/capability/capability.ts index 96819ea47..3b1c8770c 100644 --- a/packages/definitions/src/capability/capability.ts +++ b/packages/definitions/src/capability/capability.ts @@ -12,7 +12,6 @@ import type { import type { CallEnded, CallStarted } from '@on.auto/tool-servers'; import { Effect, type Schema } from 'effect'; -import { composedOverview, type OverviewParts, type ToldParts } from '../overview/definition-overview.ts'; import type { Trigger } from '../registry/definition-triggers.ts'; import { noFacts, type CapabilityFacts } from '../run-work/declared-facts.ts'; import type { CancelRequestKind } from '../runs/run-events.ts'; @@ -22,7 +21,6 @@ import { noStrippedForms, type StrippedForms } from './stripped-forms.ts'; export interface DefinitionSummary { readonly description?: string; - readonly about?: string; readonly inputSchema?: Schema.JsonObject; readonly outputSchema?: Schema.JsonObject; readonly warnings?: readonly string[]; @@ -115,10 +113,6 @@ export interface AnsweringRun { export type AnswerIssues = (answer: Schema.Json, run: AnsweringRun) => readonly Issue[]; -export type OverviewContext = Readonly<{ summary: DefinitionSummary }>; - -export type OverviewOf = (parsed: Parsed, name: string, context: OverviewContext) => OverviewParts | ToldParts; - const defaultLongestAnyRunMs = 600_000; export interface CapabilityDeclaration { @@ -135,7 +129,6 @@ export interface CapabilityDeclaration { saved: SavedDefinitions, ) => Effect.Effect; readonly summarize: (parsed: NoInfer) => DefinitionSummary; - readonly overview?: OverviewOf>; readonly run: ( parsed: NoInfer, input: Schema.Json, @@ -162,7 +155,6 @@ export interface CapabilityDeclaration { export interface PreparedDefinition { readonly summary: DefinitionSummary; - readonly overview: (name: string) => ReturnType; readonly check: (saved: SavedDefinitions) => Effect.Effect; readonly run: (input: Schema.Json, context: RunContext) => Effect.Effect; readonly whenCancelled: WhenCancelled; @@ -263,7 +255,7 @@ function declaredRuns(definition: CapabilityDeclaration, longest } export function defineCapability(definition: CapabilityDeclaration): Capability { - const { type, title, guide, noun, describeOutput, mediaType, parse, summarize, overview, run } = definition; + const { type, title, guide, noun, describeOutput, mediaType, parse, summarize, run } = definition; const bounds = declaredBounds(definition); const { check, whenCancelled, callsTools, answersItsShapeFor, finishesLaterFor, longestRunOf, longestRunFor } = declaredRuns(definition, bounds.longestAnyRunMs); @@ -280,21 +272,17 @@ export function defineCapability(definition: CapabilityDeclaration parse(source).pipe( - Effect.map((parsed) => { - const summary = summarize(parsed); - return { - summary, - overview: (name) => composedOverview({ name, noun, summary, parts: overview?.(parsed, name, { summary }) }), - check: (saved) => Effect.suspend(() => check(parsed, source, saved)), - run: (input, context) => run(parsed, input, context, stripped), - whenCancelled, - callsTools: callsTools(parsed), - finishesLaterFor: (input) => finishesLaterFor(parsed, input), - answersItsShapeFor: (input) => answersItsShapeFor(parsed, input), - longestRunMs: longestRunOf(parsed), - longestRunMsFor: (input) => longestRunFor(parsed, input), - }; - }), + Effect.map((parsed) => ({ + summary: summarize(parsed), + check: (saved) => Effect.suspend(() => check(parsed, source, saved)), + run: (input, context) => run(parsed, input, context, stripped), + whenCancelled, + callsTools: callsTools(parsed), + finishesLaterFor: (input) => finishesLaterFor(parsed, input), + answersItsShapeFor: (input) => answersItsShapeFor(parsed, input), + longestRunMs: longestRunOf(parsed), + longestRunMsFor: (input) => longestRunFor(parsed, input), + })), ), }; } diff --git a/packages/definitions/src/checking/schema-refusals.test.ts b/packages/definitions/src/checking/schema-refusals.test.ts index ed814ffcd..5cf299986 100644 --- a/packages/definitions/src/checking/schema-refusals.test.ts +++ b/packages/definitions/src/checking/schema-refusals.test.ts @@ -63,7 +63,7 @@ describe('the words of the rule a value broke', () => { }); describe('the words of a value of another kind', () => { - it('says what a field of another kind takes, as the overview words the field', () => { + it('says what a field of another kind takes, in the words of its schema', () => { expect([ fieldWords({ type: 'integer', minimum: 1, maximum: 100 }, 2.5), fieldWords({ type: 'number' }, 'many'), diff --git a/packages/definitions/src/checking/schema-refusals.ts b/packages/definitions/src/checking/schema-refusals.ts index f2c7ee758..8cd784428 100644 --- a/packages/definitions/src/checking/schema-refusals.ts +++ b/packages/definitions/src/checking/schema-refusals.ts @@ -9,9 +9,8 @@ import { articled, asSentence, InvalidInput, type Issue } from '@on.auto/operati import { measureOf, mostValueDepth } from '@on.auto/workflow-engine'; import { Effect, JsonPointer, type Schema } from 'effect'; -import { plainTextOf } from '../overview/markdown-text.ts'; -import { trailingWords } from '../overview/schema-bounds.ts'; -import { forms } from '../overview/schema-nouns.ts'; +import { trailingWords } from '../schema-words/schema-bounds.ts'; +import { forms } from '../schema-words/schema-nouns.ts'; import { isItemOf, listOf, @@ -20,8 +19,8 @@ import { type JsonSchema, type PlaceStep, type SchemaPlace, -} from '../overview/schema-reading.ts'; -import { descriptionOf, examplesWordsOf, expectedWordsOf, givesAnExample } from '../overview/schema-words.ts'; +} from '../schema-words/schema-reading.ts'; +import { descriptionOf, examplesWordsOf, expectedWordsOf, givesAnExample } from '../schema-words/schema-words.ts'; const kindKeywords = ['type', 'enum', 'const', 'anyOf', 'oneOf', 'properties', 'items', 'prefixItems']; @@ -97,7 +96,7 @@ function refusedWords(steps: readonly PlaceStep[]): string { function kindWords(schemas: readonly JsonSchema[], root: JsonSchema, detail: string): string { const said = schemas.find((schema) => kindKeywords.some((keyword) => schema[keyword] !== undefined)); - return said === undefined ? asPhrase(detail) : `expected ${plainTextOf(expectedWordsOf(said, root))}`; + return said === undefined ? asPhrase(detail) : `expected ${expectedWordsOf(said, root)}`; } function holderSentence({ document, what }: CompiledSchema, tokens: readonly string[], breach: Breach): string { diff --git a/packages/definitions/src/document.ts b/packages/definitions/src/document.ts index c54bde657..b241cb070 100644 --- a/packages/definitions/src/document.ts +++ b/packages/definitions/src/document.ts @@ -6,5 +6,4 @@ export { type FrontMatterShape, type ReadFrontMatter, } from './document/front-matter-keys.ts'; -export { AboutSchema, DescriptionSchema, type ValueContract } from './document/function-fields.ts'; -export { writtenTextIssues, type TextPosition } from './document/written-texts.ts'; +export { DescriptionSchema, type ValueContract } from './document/function-fields.ts'; diff --git a/packages/definitions/src/document/front-matter-keys.ts b/packages/definitions/src/document/front-matter-keys.ts index 16a7a2178..9e2ced21f 100644 --- a/packages/definitions/src/document/front-matter-keys.ts +++ b/packages/definitions/src/document/front-matter-keys.ts @@ -3,7 +3,6 @@ import { Predicate, Result, SchemaIssue, type Schema, type StandardSchema } from import { issueAt, type DocumentIssue, type SourceLines } from './document-issue.ts'; import { readFrontMatter } from './front-matter-reading.ts'; -import { describingPointers, frontMatterTextIssues } from './written-texts.ts'; export interface FrontMatterSection { readonly name: string | undefined; @@ -75,16 +74,9 @@ export function frontMatterIn( firstLine: number, shape: FrontMatterShape, ): Result.Result { - return Result.map( - readFrontMatter(text, firstLine, shape.required, new Set(describingPointers)), - ({ value, lines, positions }) => ({ - root: value, - lines, - issues: [ - ...unknownKeyIssues(value, lines, shape.sections), - ...typeIssues(value, lines, shape), - ...frontMatterTextIssues({ root: value, positions }, { lines: text.split('\n'), firstLine }), - ], - }), - ); + return Result.map(readFrontMatter(text, firstLine, shape.required), ({ value, lines }) => ({ + root: value, + lines, + issues: [...unknownKeyIssues(value, lines, shape.sections), ...typeIssues(value, lines, shape)], + })); } diff --git a/packages/definitions/src/document/front-matter-reading.ts b/packages/definitions/src/document/front-matter-reading.ts index e354ff975..dd31ac341 100644 --- a/packages/definitions/src/document/front-matter-reading.ts +++ b/packages/definitions/src/document/front-matter-reading.ts @@ -13,12 +13,10 @@ import { } from 'yaml'; import type { DocumentIssue, SourceLines } from './document-issue.ts'; -import type { TextPosition } from './written-texts.ts'; export interface FrontMatterReading { readonly value: Schema.JsonObject; readonly lines: SourceLines; - readonly positions: ReadonlyMap; } const frontMatterNesting = 72; @@ -36,7 +34,6 @@ interface Place { interface Reading { readonly report: (line: number, pointer: string, detail: string) => void; readonly locate: (pointer: string, line: number) => void; - readonly position: (pointer: string, node: unknown) => void; readonly lineOf: (node: unknown, fallback: number) => number; } @@ -97,9 +94,6 @@ function valueOf(node: unknown, place: Place, reading: Reading): Schema.Json { reading.report(line, place.pointer, 'Expected text, a finite number, true, false or null'); return null; } - if (typeof value === 'string') { - reading.position(place.pointer, node); - } return value; } @@ -146,7 +140,6 @@ export function readFrontMatter( text: string, firstLine: number, required: string, - positioned: ReadonlySet, ): Result.Result { const lineCounter = new LineCounter(); const document = parseDocument(text, { ...parseOptions, lineCounter }); @@ -167,7 +160,6 @@ export function readFrontMatter( } const issues: DocumentIssue[] = []; const lines = new Map([['', firstLine]]); - const positions = new Map(); const value = objectOf( () => contents, { pointer: '', depth: 0, line: firstLine }, @@ -178,17 +170,11 @@ export function readFrontMatter( locate: (pointer, line) => { lines.set(pointer, line); }, - position: (pointer, node) => { - const offset = offsetOf(node); - if (offset !== undefined && positioned.has(pointer)) { - positions.set(pointer, { line: lineAt(offset), column: lineCounter.linePos(offset).col }); - } - }, lineOf: (node, fallback) => { const offset = offsetOf(node); return offset === undefined ? fallback : lineAt(offset); }, }, ); - return issues.length > 0 ? Result.fail(issues) : Result.succeed({ value, lines, positions }); + return issues.length > 0 ? Result.fail(issues) : Result.succeed({ value, lines }); } diff --git a/packages/definitions/src/document/front-matter.test.ts b/packages/definitions/src/document/front-matter.test.ts index 7c66e48e3..a9921c8d2 100644 --- a/packages/definitions/src/document/front-matter.test.ts +++ b/packages/definitions/src/document/front-matter.test.ts @@ -4,7 +4,6 @@ import { describe, expect, it } from 'vitest'; import { issueText, reportedIssues } from './document-issue.ts'; import { splitDocument } from './document-split.ts'; import { frontMatterIn, type FrontMatterShape, type ReadFrontMatter } from './front-matter-keys.ts'; -import { AboutSchema, DescriptionSchema } from './function-fields.ts'; const NoteSchema = Schema.Struct({ title: Schema.String, @@ -27,37 +26,22 @@ const note: FrontMatterShape = { required: 'the title', }; -const DescribedSchema = Schema.Struct({ - description: Schema.optionalKey(DescriptionSchema), - about: Schema.optionalKey(AboutSchema), -}); - -const described: FrontMatterShape = { - sections: [{ name: undefined, keys: Object.keys(DescribedSchema.fields) }], - decode: Schema.decodeUnknownResult(DescribedSchema, { errors: 'all' }), - required: 'what it is for', -}; - -function read(source: string, shape = note): Result.Result { +function read(source: string): Result.Result { return Result.mapError( Result.flatMap(splitDocument(source, 'A note'), ({ frontMatter, frontMatterLine }) => - frontMatterIn(frontMatter, frontMatterLine, shape), + frontMatterIn(frontMatter, frontMatterLine, note), ), (issues) => reportedIssues(issues).map((issue) => issueText(issue)), ); } -function issuesIn(source: string, shape = note): readonly string[] { - return Result.match(read(source, shape), { +function issuesIn(source: string): readonly string[] { + return Result.match(read(source), { onSuccess: ({ issues }) => reportedIssues(issues).map((issue) => issueText(issue)), onFailure: (issues) => issues, }); } -function headingRefusalAt(line: number, key: string, where = 'this line'): string { - return `Line ${line}, /${key}: ${key} takes no heading, since an overview writes its own: ${where} makes one, with # before its words or as a line of = or - under the words above it`; -} - function rootOf(source: string): Schema.JsonObject { return Result.getOrThrow(read(source)).root; } @@ -169,43 +153,3 @@ describe('the keys of the front matter', () => { expect(issuesIn(documentOf(`title: A\n${many}`)).at(-1)).toBe('Line 23: 3 more issues are not shown'); }); }); - -describe('the description and the about of the front matter', () => { - it('take no heading, each refused at its own line in a literal block', () => { - const source = documentOf( - ['description: |', ' Reviews a brief.', ' ## When', 'about: |-', '', ' Who it serves', ' ---'].join('\n'), - ); - - expect(issuesIn(source, described)).toEqual([headingRefusalAt(4, 'description'), headingRefusalAt(8, 'about')]); - }); - - it('take no heading in any other style, refused at the line their value opens, naming the line by its first words', () => { - expect( - issuesIn(documentOf("description: '# Reviews a brief'\nabout: >\n Who it serves.\n\n # When"), described), - ).toEqual([ - headingRefusalAt(2, 'description', 'the line that begins “# Reviews a brief”'), - headingRefusalAt(3, 'about', 'the line that begins “# When”'), - ]); - expect(issuesIn(documentOf('title: "# A note"\nextra:\n anything: "# kept"'))).toEqual([]); - }); - - it('leave no block open, and hold no line ## Details, each refused at its line', () => { - const fence = '`'.repeat(3); - - expect( - issuesIn(documentOf(['about: |', ' Who it serves.', ` ${fence}`, ' ## Details'].join('\n')), described), - ).toEqual([ - 'Line 4, /about: about leaves open the fence that this line opens, so the rest of the overview would be read inside it: close it', - "Line 5, /about: about may not hold the line ## Details, which is the overview's own, since the page that shows an overview splits it at that text: this line is one", - ]); - }); - - it('take Markdown with paragraphs, lists and emphasis', () => { - const about = ['about: |', ' It is for the *team*.', '', ' - Who it serves', ' - When it runs'].join('\n'); - - expect(Result.getOrThrow(read(documentOf(about), described))).toMatchObject({ - root: { about: 'It is for the *team*.\n\n- Who it serves\n- When it runs\n' }, - issues: [], - }); - }); -}); diff --git a/packages/definitions/src/document/function-fields.test.ts b/packages/definitions/src/document/function-fields.test.ts index bd970c5df..6691385ed 100644 --- a/packages/definitions/src/document/function-fields.test.ts +++ b/packages/definitions/src/document/function-fields.test.ts @@ -1,12 +1,10 @@ import { Schema } from 'effect'; import { describe, expect, it } from 'vitest'; -import { AboutSchema, DescriptionSchema } from '../document.ts'; +import { DescriptionSchema } from '../document.ts'; const holds = Schema.is(DescriptionSchema); -const holdsAbout = Schema.is(AboutSchema); - describe('the description of a function document', () => { it('takes 1 to 1,000 characters', () => { expect([holds(''), holds('x'), holds('x'.repeat(1000)), holds('x'.repeat(1001))]).toEqual([ @@ -17,15 +15,3 @@ describe('the description of a function document', () => { ]); }); }); - -describe('the about of a function document', () => { - it('takes 1 to 4,000 characters, counted as UTF-16 units', () => { - expect([holdsAbout(''), holdsAbout('x'), holdsAbout('x'.repeat(4000)), holdsAbout('x'.repeat(4001))]).toEqual([ - false, - true, - true, - false, - ]); - expect([holdsAbout('😀'.repeat(2000)), holdsAbout(`${'😀'.repeat(2000)}x`)]).toEqual([true, false]); - }); -}); diff --git a/packages/definitions/src/document/function-fields.ts b/packages/definitions/src/document/function-fields.ts index 95a8d43ea..e07371af2 100644 --- a/packages/definitions/src/document/function-fields.ts +++ b/packages/definitions/src/document/function-fields.ts @@ -3,12 +3,8 @@ import { Schema } from 'effect'; const descriptionLength = 1000; -const aboutLength = 4000; - export const DescriptionSchema = Schema.String.check(Schema.isMinLength(1), Schema.isMaxLength(descriptionLength)); -export const AboutSchema = Schema.String.check(Schema.isMinLength(1), Schema.isMaxLength(aboutLength)); - export interface ValueContract { readonly schema?: CompiledSchema; } diff --git a/packages/definitions/src/document/written-texts.test.ts b/packages/definitions/src/document/written-texts.test.ts deleted file mode 100644 index 98d0dd036..000000000 --- a/packages/definitions/src/document/written-texts.test.ts +++ /dev/null @@ -1,192 +0,0 @@ -import { describe, expect, it } from 'vitest'; - -import { writtenTextIssues } from '../document.ts'; - -type Finding = readonly [number, 'heading' | 'details' | 'fence' | 'html']; - -const literalHeader = { line: 1, column: 8 }; - -function kindOf(detail: string): Finding[1] { - if (detail.includes('takes no heading')) { - return 'heading'; - } - if (detail.includes('leaves open the fence')) { - return 'fence'; - } - return detail.includes('leaves open the HTML block') ? 'html' : 'details'; -} - -function findingsOf(...lines: readonly string[]): readonly Finding[] { - const source = ['about: |', ...lines.map((line) => ` ${line}`)]; - return writtenTextIssues(lines.join('\n'), 'about', { lines: source, firstLine: 1 }, literalHeader).map( - ({ line, detail }): Finding => [line - 2, kindOf(detail)], - ); -} - -interface Placed { - readonly line: number; - readonly column: number; - readonly detail: string; -} - -function placesOf( - issues: readonly Placed[], -): readonly { readonly line: number; readonly column: number; readonly where: string | undefined }[] { - return issues.map(({ line, column, detail }) => ({ - line, - column, - where: /: (?the line that begins “.+”) makes one/u.exec(detail)?.groups?.['where'], - })); -} - -function aboutIssuesAt(line: number, text: string, lines: readonly string[]): readonly Placed[] { - return writtenTextIssues(text, 'about', { lines, firstLine: line }, { line, column: 8 }); -} - -function headingsOf(...lines: readonly string[]): readonly number[] { - return findingsOf(...lines) - .filter(([, kind]) => kind === 'heading') - .map(([index]) => index); -} - -describe('a refusal of a text of a document', () => { - it('is at the line of its own in a literal block, and at the column its text starts at, in words that name its key', () => { - const source = ['---', 'description: |', ' Who it serves.', '', ' ## When it runs']; - - expect( - writtenTextIssues( - 'Who it serves.\n\n## When it runs\n', - 'description', - { lines: source, firstLine: 1 }, - { line: 2, column: 14 }, - ), - ).toEqual([ - { - line: 5, - column: 5, - detail: - 'description takes no heading, since an overview writes its own: this line makes one, with # before its words or as a line of = or - under the words above it', - }, - ]); - }); - - it('is at the line and the column its value opens at in any other style, naming the line by its first words', () => { - const folded = ['about: >', ' Intro.', '', ' ## Heading of the team']; - const quoted = ['about: "Intro.\\n ## Heading\\tof the brief"']; - - expect([ - placesOf(aboutIssuesAt(3, 'Intro.\n## Heading of the team\n', folded)), - placesOf(aboutIssuesAt(7, 'Intro.\n ## Heading\tof the brief', quoted)), - ]).toEqual([ - [{ line: 3, column: 8, where: 'the line that begins “## Heading of the”' }], - [{ line: 7, column: 8, where: 'the line that begins “## Heading of the”' }], - ]); - }); - - it('reads a value on a line its source does not hold as one in no literal block', () => { - expect( - placesOf(writtenTextIssues('# Heading', 'about', { lines: [], firstLine: 1 }, { line: 4, column: 3 })), - ).toEqual([{ line: 4, column: 3, where: 'the line that begins “# Heading”' }]); - }); -}); - -describe('a heading in a text of a document', () => { - it('is a line of one to six # and a space, a tab or nothing, or a line of = or - under a line of a paragraph', () => { - expect(headingsOf('# One', 'Text', '###### Six', '#\tTabbed', '#', 'Two', '===', 'Three', '---')).toEqual([ - 0, 2, 3, 4, 6, 8, - ]); - }); - - it('is indented at most three spaces after a line of a paragraph, as a heading line or as an underline', () => { - expect([ - headingsOf('Text', ' # Three'), - headingsOf('Text', ' # Four'), - headingsOf('Text', '\t# Tabbed'), - headingsOf('Text', ' ==='), - headingsOf('Text', ' ==='), - ]).toEqual([[1], [], [], [1], []]); - }); - - it('is none of a tag, seven #, an escaped #, a thematic break or a line of = after a blank line', () => { - expect(findingsOf('#tag', '####### Seven', '\\# Escaped', '', '---', '', '===', '***', 'Text', '- - -')).toEqual( - [], - ); - }); -}); - -describe('a heading in a container of a text of a document', () => { - it('is in a quote, in a list item and in a list item of a quote, after the markers of each', () => { - expect([ - headingsOf('> # Words'), - headingsOf('- # Words'), - headingsOf('1. ## Words'), - headingsOf('> - # Words'), - ]).toEqual([[0], [0], [0], [0]]); - }); - - it('is an underline in a quote under a line of the same quote, and one indented to the content of an item under it', () => { - expect([ - headingsOf('> Title', '> ==='), - headingsOf('- item', ' ---'), - headingsOf('- a', ' - b', ' ==='), - ]).toEqual([[1], [1], [2]]); - }); - - it('is no underline under a line that opens or lazily goes on with an item or a quote', () => { - expect([ - findingsOf('- item', '---'), - findingsOf('> quote', '==='), - findingsOf('- item', 'goes on', '==='), - findingsOf('> - item', '> ---'), - findingsOf('> > deep', '> goes on', '> ==='), - ]).toEqual([[], [], [], [], []]); - }); - - it('is in an item, indented to its content with spaces or a tab, and not in indented code of the item', () => { - expect([ - headingsOf('- item', '', ' # In it'), - headingsOf('- a', ' - b', '', ' # In b'), - headingsOf('- item', '\t# In it'), - headingsOf('-\t# Words'), - headingsOf('- # Code'), - headingsOf('-', ' # In the item'), - headingsOf('- item', '>', ' # Code'), - ]).toEqual([[2], [3], [1], [0], [], [1], []]); - }); - - it('is an underline under a paragraph that an empty item or an ordered item not from 1 cannot interrupt', () => { - expect([headingsOf('Text', '-'), headingsOf('Text', '2. two', ' ===')]).toEqual([[1], []]); - }); -}); - -describe('a heading in a block of a text of a document', () => { - it('is none of a line in a fence, in indented code, in an HTML block or under a row of a table', () => { - expect( - findingsOf( - '```', - '# A comment of a shell', - '```', - '', - ' # Code', - '
', - '# Inside', - '
', - '', - '| a | b |', - '| - | - |', - '| 1 | 2 |', - '---', - ), - ).toEqual([]); - }); - - it('is a line after an HTML block that ends on its line, and after a tag that interrupts a paragraph, and in no block of a tag alone on its line', () => { - expect([ - headingsOf('', '# After it'), - headingsOf('
', '# Inside', '
', '# After it'), - headingsOf('', '# Inside', '', '# After it'), - headingsOf('Text', '', '# After it'), - headingsOf('', '# Inside'), - ]).toEqual([[1], [3], [3], [2], []]); - }); -}); diff --git a/packages/definitions/src/document/written-texts.ts b/packages/definitions/src/document/written-texts.ts deleted file mode 100644 index 23899b492..000000000 --- a/packages/definitions/src/document/written-texts.ts +++ /dev/null @@ -1,125 +0,0 @@ -import { quoted } from '@on.auto/operations'; -import type { Schema } from 'effect'; - -import { - detailsHeading, - detailsLinesIn, - headingLinesIn, - unclosedBlockIn, - type UnclosedBlock, -} from '../overview/markdown-text.ts'; -import type { DocumentIssue } from './document-issue.ts'; - -interface TextSource { - readonly lines: readonly string[]; - readonly firstLine: number; -} - -export interface TextPosition { - readonly line: number; - readonly column: number; -} - -interface TextIssue extends TextPosition { - readonly detail: string; -} - -interface Finding { - readonly index: number; - readonly opening: string; - readonly detail: (where: string) => string; -} - -export const describingPointers: readonly string[] = ['/description', '/about']; - -const literalBlock = /^\|/u; - -const blanks = /[ \t]+/u; - -const mostOpeningWords = 4; - -function openingWordsOf(line: string): string { - return line - .split(blanks) - .filter((word) => word !== '') - .slice(0, mostOpeningWords) - .join(' '); -} - -function findingsAt( - lines: readonly string[], - indexes: readonly number[], - detail: (where: string) => string, -): readonly Finding[] { - return lines.flatMap((line, index) => - indexes.includes(index) ? [{ index, opening: openingWordsOf(line), detail }] : [], - ); -} - -function detailsWords(key: string, where: string): string { - return `${key} may not hold the line ${detailsHeading}, which is the overview's own, since the page that shows an overview splits it at that text: ${where} is one`; -} - -function headingWords(key: string, where: string): string { - return `${key} takes no heading, since an overview writes its own: ${where} makes one, with # before its words or as a line of = or - under the words above it`; -} - -function unclosedWords(key: string, unclosed: UnclosedBlock | undefined, where: string): string { - const block = unclosed?.kind === 'fence' ? 'fence' : 'HTML block'; - return `${key} leaves open the ${block} that ${where} opens, so the rest of the overview would be read inside it: close it`; -} - -function findingsIn(key: string, text: string): readonly Finding[] { - const lines = text.split('\n'); - const details = detailsLinesIn(text); - const headings = headingLinesIn(text).filter((index) => !details.includes(index)); - const unclosed = unclosedBlockIn(text); - return [ - ...findingsAt(lines, details, (where) => detailsWords(key, where)), - ...findingsAt(lines, headings, (where) => headingWords(key, where)), - ...findingsAt(lines, unclosed === undefined ? [] : [unclosed.line], (where) => unclosedWords(key, unclosed, where)), - ]; -} - -function lineAt({ lines, firstLine }: TextSource, line: number): string { - return lines[line - firstLine] ?? ''; -} - -function columnOfText(line: string): number { - return line.length - line.trimStart().length + 1; -} - -function placed(findings: readonly Finding[], source: TextSource, opens: TextPosition): readonly TextIssue[] { - const literal = literalBlock.test(lineAt(source, opens.line).slice(opens.column - 1)); - return findings - .toSorted((one, other) => one.index - other.index) - .map(({ index, opening, detail }) => { - const line = opens.line + 1 + index; - return literal - ? { line, column: columnOfText(lineAt(source, line)), detail: detail('this line') } - : { line: opens.line, column: opens.column, detail: detail(`the line that begins ${quoted(opening)}`) }; - }); -} - -export function writtenTextIssues( - text: string, - key: string, - source: TextSource, - opens: TextPosition, -): readonly TextIssue[] { - return placed(findingsIn(key, text), source, opens); -} - -export function frontMatterTextIssues( - { root, positions }: { readonly root: Schema.JsonObject; readonly positions: ReadonlyMap }, - source: TextSource, -): readonly DocumentIssue[] { - return describingPointers.flatMap((pointer) => { - const key = pointer.slice(1); - const text = root[key]; - const opens = positions.get(pointer); - return typeof text === 'string' && opens !== undefined - ? writtenTextIssues(text, key, source, opens).map(({ line, detail }) => ({ line, pointer, detail })) - : []; - }); -} diff --git a/packages/definitions/src/events/playbook-facts.test.ts b/packages/definitions/src/events/playbook-facts.test.ts index 257d70fbd..1ee9f3419 100644 --- a/packages/definitions/src/events/playbook-facts.test.ts +++ b/packages/definitions/src/events/playbook-facts.test.ts @@ -11,7 +11,6 @@ import { type RunEvent, } from '../index.ts'; import { runStreamNameOf } from '../runs/run-decider.ts'; -import { overviewTitled } from '../testing/overview-sections.ts'; import { testRunId as runId } from '../testing/run-facts.ts'; const recordId = '0b1c2d3e-4f50-5a6b-8c7d-8e9fa0b1c2d3'; @@ -127,7 +126,6 @@ describe('the records of a run that are no facts', () => { const content = { source: '---\nmodel: anthropic/claude-sonnet-4-5\n---\nSummarize.', - overview: overviewTitled('summary'), }; describe('the facts of a definition as events', () => { diff --git a/packages/definitions/src/index.ts b/packages/definitions/src/index.ts index 1b2be9b56..b9e09d3e9 100644 --- a/packages/definitions/src/index.ts +++ b/packages/definitions/src/index.ts @@ -11,8 +11,6 @@ export { defineCapability, type AnswerIssues, type AnsweringRun, - type OverviewContext, - type OverviewOf, defaultRunWords, type CancelDecision, type CancelledRun, @@ -176,7 +174,6 @@ export { } from './registry/definition-triggers.ts'; export { runStartedOf } from './runs/run-starts.ts'; export { inWords, nameInWords, outputInWords } from './plain-language/in-words.ts'; -export { charactersIn } from './plain-language/characters.ts'; export { triggerNamed } from './plain-language/event-words.ts'; export { mostInputBytes, mostInputDepth, mostResultBytes } from './runs/recorded-size.ts'; export { @@ -207,86 +204,11 @@ export { } from './capability/function-terminology.ts'; export { checkedInput, refusedByTheInputSchema, refusedInput, wordedIssuesOf } from './checking/schema-refusals.ts'; export { checkedWorker } from './checking/worker-url.ts'; -export { - bulletList, - definitionPhrase, - listedPhrases, - mostTableRows, - named, - numberedList, - paragraph, - sectionOf, - type BulletList, - type Diagram, - type DiagramEdge, - type DiagramGroup, - type DiagramNode, - type FieldRow, - type FieldTable, - type NodeShape, - type NumberedList, - type Overview, - type OverviewBlock, - type OverviewSection, - type Paragraph, - type Phrase, - type PhrasedDefinition, - type Phrases, - type Quote, -} from './overview/overview.ts'; -export { durationInWords } from './overview/duration-words.ts'; -export { eventFilterPhrases, type EventFilter } from './overview/event-filter-words.ts'; -export { markdownOf, mostOverviewBytes } from './overview/markdown-writer.ts'; -export { describedSchemaBlocks, mostFieldDescriptionCharacters, schemaBlocks } from './overview/schema-words.ts'; -export { plainTextOf } from './overview/markdown-text.ts'; -export { isJsonSchema } from './overview/schema-reading.ts'; -export { callSection, callSentence } from './overview-words/call-words.ts'; -export { computedSentence, programSection } from './overview-words/computation-words.ts'; -export { valueMeaning } from './overview-words/field-meanings.ts'; -export { - answerSection, - answerSentence, - answerWaitSentence, - inboxSentence, - messageLead, - notTakenSentence, - notificationSentence, - partyNamedUnderDetails, - replyRuleSentence, - replyWaitSentence, - unansweredSentence, - whereSection, - type ReplyParts, -} from './overview-words/request-words.ts'; -export { replyDiagram } from './overview-words/reply-diagram-words.ts'; -export { shownUnderSource } from './overview-words/source-words.ts'; -export { templateLead } from './overview-words/template-words.ts'; -export { theToolOf } from './overview-words/tool-words.ts'; -export { undescribedSentence } from './overview-words/value-words.ts'; export type { OutputShape } from './shapes/output-shape.ts'; export { OutputShapeNameSchema, type OutputShapeName } from './shapes/shape-names.ts'; export { outputShapeNamed, outputShapeNames, shapeMismatchOf } from './shapes/output-shapes.ts'; export { declaredShape, type ShapeRefusal } from './shapes/shape-declarations.ts'; export { publishedOutputOf } from './shapes/published-output.ts'; -export { seriesBounds, seriesMarksSchema, seriesShape } from './shapes/series-shape.ts'; -export { periodNouns, seriesPathBlocks } from './shapes/shape-words.ts'; +export { seriesMarksSchema, seriesShape } from './shapes/series-shape.ts'; export { rangeHolds } from './shapes/shape-values.ts'; -export { - composedOverview, - inputDetails, - inputSection, - outputDetails, - outputSection, - type OverviewParts, - type ToldParts, -} from './overview/definition-overview.ts'; -export { overviewAtRead, type ReadKnowledge } from './used-by/overview-at-read.ts'; -export type { DefinitionCalled, DefinitionUser } from './used-by/user-relations.ts'; -export { - CalledDefinitionSchema, - type CalledDefinition, - type CallingStep, - type CallRole, - type NamedDefinition, - type UserRelation, -} from './used-by/called-definitions.ts'; +export type { CalledDefinition, CallingStep, CallRole } from './used-by/called-definitions.ts'; diff --git a/packages/definitions/src/markdown/html-blocks.ts b/packages/definitions/src/markdown/html-blocks.ts deleted file mode 100644 index 8ad8713a5..000000000 --- a/packages/definitions/src/markdown/html-blocks.ts +++ /dev/null @@ -1,56 +0,0 @@ -export type HtmlBlock = 'raw' | 'comment' | 'instruction' | 'declaration' | 'data' | 'element' | 'tag'; - -const blankLine = /^[ \t]*$/u; - -const blockTags = [ - ['address', 'article', 'aside', 'base', 'basefont', 'blockquote', 'body', 'caption', 'center', 'col'], - ['colgroup', 'dd', 'details', 'dialog', 'dir', 'div', 'dl', 'dt', 'fieldset', 'figcaption', 'figure'], - ['footer', 'form', 'frame', 'frameset', 'h[1-6]', 'head', 'header', 'hr', 'html', 'iframe', 'legend'], - ['li', 'link', 'main', 'menu', 'menuitem', 'nav', 'noframes', 'ol', 'optgroup', 'option', 'p', 'param'], - ['search', 'section', 'summary', 'table', 'tbody', 'td', 'tfoot', 'th', 'thead', 'title', 'tr', 'track', 'ul'], -].flat(); - -const tagName = '[A-Za-z][A-Za-z0-9-]*'; - -const attribute = `[ \\t]+[A-Za-z_:][A-Za-z0-9_.:-]*(?:[ \\t]*=[ \\t]*(?:[^ \\t"'=<>\`]+|'[^']*'|"[^"]*"))?`; - -const htmlBlocks: readonly HtmlBlock[] = ['raw', 'comment', 'instruction', 'declaration', 'data', 'element', 'tag']; - -const htmlOpeners: Readonly> = { - raw: /^<(?:script|pre|style|textarea)(?:[ \t>]|$)/iu, - comment: /^', - instruction: '?>', - declaration: '>', - data: ']]>', - element: '', - tag: '', -}; - -const endOfTheTagName = /[ \t>]|$/u; - -export const htmlEnds: Readonly> = { - raw: /<\/(?:script|pre|style|textarea)>/iu, - comment: /--!?>/u, - instruction: /\?>/u, - declaration: />/u, - data: /\]\]>/u, - element: blankLine, - tag: blankLine, -}; - -export function htmlOpenedBy(content: string, interrupting: boolean): HtmlBlock | undefined { - return htmlBlocks.find((html) => htmlOpeners[html].test(content) && !(html === 'tag' && interrupting)); -} - -export function htmlCloserOf(html: HtmlBlock, opener: string): string { - return html === 'raw' ? `` : htmlClosers[html]; -} diff --git a/packages/definitions/src/markdown/markdown-lines.ts b/packages/definitions/src/markdown/markdown-lines.ts deleted file mode 100644 index bcb732735..000000000 --- a/packages/definitions/src/markdown/markdown-lines.ts +++ /dev/null @@ -1,298 +0,0 @@ -import { htmlCloserOf, htmlEnds, htmlOpenedBy, type HtmlBlock } from './html-blocks.ts'; -import { tableLinesOf } from './table-rows.ts'; - -export interface OpenBlock { - readonly kind: 'fence' | HtmlBlock; - readonly closer: string; - readonly line: number; - readonly within: number | undefined; -} - -type Form = 'atx' | 'underline' | 'break'; - -interface ReadLine { - readonly prose: boolean; - readonly text: boolean; - readonly form: Form | undefined; - readonly separates: boolean; - readonly container: string; - readonly quoted: number; -} - -interface Reading { - readonly open: OpenBlock | undefined; - readonly codeMayStart: boolean; - readonly inCode: boolean; - readonly items: readonly number[]; - readonly startedEmpty: boolean; - readonly last: ReadLine | undefined; -} - -interface Stripped { - readonly quoted: number; - readonly opens: readonly number[]; - readonly at: number; - readonly rest: string; - readonly container: string; -} - -interface Place { - readonly quoted: number; - readonly kept: readonly number[]; - readonly opens: readonly number[]; - readonly container: string; - readonly indent: number; - readonly content: string; -} - -interface Step { - readonly reading: Reading; - readonly read: ReadLine; -} - -interface BlockLine { - readonly block: boolean; - readonly open: OpenBlock | undefined; -} - -export interface MarkdownLines { - readonly reads: readonly ReadLine[]; - readonly open: OpenBlock | undefined; -} - -const blankLine = /^[ \t]*$/u; - -const leadingBlanks = /^[ \t]+/u; - -const fenceOpener = /^(?:`{3,}(?=[^`]*$)|~{3,})/u; - -const listMarker = /^(?:[-+*]|\d{1,9}[.)])(?=[ \t]|$)/u; - -const interruptingItem = /^(?:[-+*]|0*1[.)])[ \t]+[^ \t]/u; - -const quoteMarker = /^ {0,3}> ?/u; - -const atxHeading = /^#{1,6}(?:[ \t]|$)/u; - -const thematicBreak = /^(?:(?:\*[ \t]*){3,}|(?:-[ \t]*){3,}|(?:_[ \t]*){3,})$/u; - -const setextUnderline = /^(?:=+|-+)[ \t]*$/u; - -const atRest: Reading = { - open: undefined, - codeMayStart: true, - inCode: false, - items: [], - startedEmpty: false, - last: undefined, -}; - -function blanksOf(line: string): string { - return line.slice(0, line.length - line.replace(leadingBlanks, '').length); -} - -function withoutBlanks(line: string): string { - return line.slice(blanksOf(line).length); -} - -function columnsOf(blanks: string, from = 0): number { - return blanks - .split('') - .reduce((column, character) => (character === '\t' ? column + 4 - (column % 4) : column + 1), from); -} - -function fromColumn(line: string, column: number): string { - let reached = 0; - let index = 0; - while (reached < column && (line[index] === ' ' || line[index] === '\t')) { - reached = columnsOf(line.charAt(index), reached); - index += 1; - } - return line.slice(index); -} - -function opensAnItem(rest: string, interrupting: boolean): boolean { - const content = withoutBlanks(rest); - const item = listMarker.test(content) && !thematicBreak.test(content); - return columnsOf(blanksOf(rest)) <= 3 && item && (!interrupting || interruptingItem.test(content)); -} - -function itemOf(rest: string, at: number): Pick { - const marker = rest.length - rest.replace(listMarker, '').length; - const spaces = blanksOf(rest.slice(marker)); - const width = columnsOf(spaces, at + marker) - at - marker; - const taken = rest.length === marker + spaces.length || width > 4 ? 1 : width; - return { at: at + marker + taken, rest: rest.slice(marker + Math.min(spaces.length, taken)) }; -} - -function strippedOnce(stripped: Stripped, paragraph: string | undefined): Stripped | undefined { - const { rest, at, quoted } = stripped; - const quote = quoteMarker.exec(rest)?.[0]; - if (quote !== undefined) { - return { ...stripped, quoted: quoted + 1, rest: rest.slice(quote.length), container: `${stripped.container}>` }; - } - if (!opensAnItem(rest, paragraph === stripped.container)) { - return undefined; - } - const item = itemOf(withoutBlanks(rest), columnsOf(blanksOf(rest), at)); - const opens = quoted === 0 ? [...stripped.opens, item.at] : stripped.opens; - return { ...stripped, ...item, opens, container: `${stripped.container}-${item.at}` }; -} - -function placeOf(line: string, items: readonly number[], paragraph: string | undefined): Place { - const kept = items.filter((column) => column <= columnsOf(blanksOf(line))); - const at = kept.at(-1) ?? 0; - const container = kept.map((column) => `-${column}`).join(''); - let stripped: Stripped = { quoted: 0, opens: [], at, rest: fromColumn(line, at), container }; - for (let next = strippedOnce(stripped, paragraph); next !== undefined; next = strippedOnce(next, paragraph)) { - stripped = next; - } - const { quoted, opens, rest } = stripped; - const content = withoutBlanks(rest); - return { quoted, kept, opens, container: stripped.container, indent: columnsOf(blanksOf(rest)), content }; -} - -function opensABlock(content: string): boolean { - const opensAContainer = listMarker.test(content) || quoteMarker.test(content); - const standsAlone = atxHeading.test(content) || thematicBreak.test(content) || fenceOpener.test(content); - return opensAContainer || standsAlone || htmlOpenedBy(content, true) !== undefined; -} - -export function standsApart(line: string): boolean { - const content = withoutBlanks(line); - return opensABlock(content) || setextUnderline.test(content); -} - -function isParagraphText(place: Place): boolean { - return place.opens.length === 0 && (place.indent > 3 || !opensABlock(place.content)); -} - -function lazilyAfter(last: ReadLine | undefined, place: Place, items: readonly number[]): ReadLine | undefined { - if (last?.prose !== true || place.quoted > last.quoted || !isParagraphText(place)) { - return undefined; - } - return place.quoted < last.quoted || place.kept.length < items.length ? last : undefined; -} - -function lineOf({ quoted }: Pick, container: string, kind: Form | 'prose' | 'other'): ReadLine { - const form = kind === 'prose' || kind === 'other' ? undefined : kind; - return { prose: kind === 'prose', text: kind === 'prose', form, separates: false, container, quoted }; -} - -function htmlLineOf(content: string, interrupting: boolean, at: Omit): BlockLine { - const html = htmlOpenedBy(content, interrupting); - if (html === undefined) { - return { block: false, open: undefined }; - } - const open = { kind: html, closer: htmlCloserOf(html, content), ...at }; - return { block: true, open: htmlEnds[html].test(content) ? undefined : open }; -} - -function blockLineOf(place: Place, interrupting: boolean, at: Omit): BlockLine { - if (place.indent > 3 || place.quoted > 0) { - return { block: false, open: undefined }; - } - const closer = fenceOpener.exec(place.content)?.[0]; - return closer === undefined - ? htmlLineOf(place.content, interrupting, at) - : { block: true, open: { kind: 'fence', closer, ...at } }; -} - -function stepOf(reading: Reading, read: ReadLine, changes: Partial): Step { - return { reading: { ...reading, startedEmpty: false, ...changes, last: read }, read }; -} - -function formOf({ content }: Place, underlines: boolean): Form | undefined { - if (atxHeading.test(content)) { - return 'atx'; - } - if (underlines && setextUnderline.test(content)) { - return 'underline'; - } - return thematicBreak.test(content) ? 'break' : undefined; -} - -function kindOf(place: Place, block: boolean, underlines: boolean): Form | 'prose' | 'other' { - if (block) { - return 'other'; - } - return (place.indent <= 3 ? formOf(place, underlines) : undefined) ?? 'prose'; -} - -function isUnderAParagraphOf(last: ReadLine | undefined, container: string): boolean { - return last?.prose === true && last.container === container; -} - -function containedStep(reading: Reading, place: Place, line: number): Step { - const items = [...place.kept, ...place.opens]; - const within = items.at(-1); - const { container } = place; - const codeMayStart = reading.codeMayStart || reading.inCode || place.opens.length > 0; - if (codeMayStart && place.indent >= 4 && place.quoted === 0) { - return stepOf(reading, lineOf(place, container, 'other'), { codeMayStart: true, inCode: true, items }); - } - const { block, open } = blockLineOf(place, reading.last?.text === true, { line, within }); - const kind = kindOf(place, block, isUnderAParagraphOf(reading.last, container)); - const changes = { open, codeMayStart: kind !== 'prose', inCode: false, items }; - return stepOf(reading, lineOf(place, container, kind), changes); -} - -function blankStep(reading: Reading, place: Place): Step { - const startedEmpty = place.opens.length > 0; - const going = reading.startedEmpty ? reading.items.slice(0, -1) : reading.items; - const items = startedEmpty || place.quoted > 0 ? [...place.kept, ...place.opens] : going; - return stepOf( - reading, - { ...lineOf(place, '', 'other'), separates: true }, - { codeMayStart: true, items, startedEmpty }, - ); -} - -function readOutside(reading: Reading, line: string, index: number): Step { - const { last } = reading; - const place = placeOf(line, reading.items, last?.prose === true ? last.container : undefined); - if (blankLine.test(place.content)) { - return blankStep(reading, place); - } - const above = lazilyAfter(reading.last, place, reading.items); - if (above !== undefined) { - const read = lineOf(above, above.container, 'prose'); - return stepOf(reading, read, { codeMayStart: false, inCode: false }); - } - return containedStep(reading, place, index); -} - -function closesFence({ closer, within }: OpenBlock, line: string): boolean { - const rest = fromColumn(line, within ?? 0); - const content = withoutBlanks(rest); - const closing = fenceOpener.exec(content)?.[0] ?? ''; - const fits = closing.startsWith(closer.charAt(0)) && closing.length >= closer.length; - return columnsOf(blanksOf(rest)) <= 3 && fits && blankLine.test(content.slice(closing.length)); -} - -function readInside(reading: Reading, open: OpenBlock, line: string): Step | undefined { - if (open.within !== undefined && !blankLine.test(line) && columnsOf(blanksOf(line)) < open.within) { - return undefined; - } - const closes = open.kind === 'fence' ? closesFence(open, line) : htmlEnds[open.kind].test(line); - const separates = closes && blankLine.test(line); - const read: ReadLine = { prose: false, text: false, form: undefined, separates, container: '', quoted: 0 }; - return stepOf(reading, read, { open: closes ? undefined : open, codeMayStart: closes }); -} - -function readLine(reading: Reading, line: string, index: number): Step { - const inside = reading.open === undefined ? undefined : readInside(reading, reading.open, line); - return inside ?? readOutside({ ...reading, open: undefined }, line, index); -} - -export function markdownLinesOf(lines: readonly string[]): MarkdownLines { - const inTable = tableLinesOf(lines); - let reading = atRest; - const reads = lines.map((line, index) => { - const step = readLine(reading, line, index); - const read = { ...step.read, prose: step.read.prose && inTable[index] === false }; - reading = { ...step.reading, last: read }; - return read; - }); - return { reads, open: reading.open }; -} diff --git a/packages/definitions/src/markdown/table-rows.ts b/packages/definitions/src/markdown/table-rows.ts deleted file mode 100644 index a4d9a8a79..000000000 --- a/packages/definitions/src/markdown/table-rows.ts +++ /dev/null @@ -1,20 +0,0 @@ -const blankLine = /^[ \t]*$/u; - -const outerBlanks = /^[ \t]+|[ \t]+$/gu; - -const delimiterCell = /^:?-+:?$/u; - -const outerPipes = /^\||\|$/gu; - -function isDelimiterRow(line: string | undefined): boolean { - const cells = line?.replaceAll(outerBlanks, '').replaceAll(outerPipes, '').split('|') ?? []; - return line?.includes('|') === true && cells.every((cell) => delimiterCell.test(cell.replaceAll(outerBlanks, ''))); -} - -export function tableLinesOf(lines: readonly string[]): readonly boolean[] { - let within = false; - return lines.map((line, index) => { - within = within ? !blankLine.test(line) : line.includes('|') && isDelimiterRow(lines.at(index + 1)); - return within; - }); -} diff --git a/packages/definitions/src/markdown/written-blocks.test.ts b/packages/definitions/src/markdown/written-blocks.test.ts deleted file mode 100644 index 243d1bfe3..000000000 --- a/packages/definitions/src/markdown/written-blocks.test.ts +++ /dev/null @@ -1,163 +0,0 @@ -import { describe, expect, it } from 'vitest'; - -import { writtenTextIssues } from '../document.ts'; - -type Finding = readonly [number, 'heading' | 'details' | 'fence' | 'html']; - -const literalHeader = { line: 1, column: 8 }; - -function kindOf(detail: string): Finding[1] { - if (detail.includes('takes no heading')) { - return 'heading'; - } - if (detail.includes('leaves open the fence')) { - return 'fence'; - } - return detail.includes('leaves open the HTML block') ? 'html' : 'details'; -} - -function findingsOf(...lines: readonly string[]): readonly Finding[] { - const source = ['about: |', ...lines.map((line) => ` ${line}`)]; - return writtenTextIssues(lines.join('\n'), 'about', { lines: source, firstLine: 1 }, literalHeader).map( - ({ line, detail }): Finding => [line - 2, kindOf(detail)], - ); -} - -describe('a fence in a text of a document', () => { - it('opens indented three spaces after a line of a paragraph, and is a line of the paragraph indented four', () => { - expect([findingsOf('Text', ' ```', '# Inside'), findingsOf('Text', ' ```', '# After it')]).toEqual([ - [[1, 'fence']], - [[2, 'heading']], - ]); - }); - - it('closes on a line of its marker indented at most three spaces, with only spaces or tabs after it', () => { - expect([ - findingsOf('```', 'code', ' ```', '# After it'), - findingsOf('```', 'code', ' ```', '# Inside'), - findingsOf('```', 'code', '```\t ', '# After it'), - findingsOf('```', 'code', '```\u00A0', '# Inside'), - ]).toEqual([[[3, 'heading']], [[0, 'fence']], [[3, 'heading']], [[0, 'fence']]]); - }); - - it('of backticks takes no backtick after its marker, so a line that holds one is a line of a paragraph', () => { - expect([findingsOf('``` a`b', '# After it', '```'), findingsOf('~~~ a`b', '# Inside', '~~~')]).toEqual([ - [ - [1, 'heading'], - [2, 'fence'], - ], - [], - ]); - }); -}); - -describe('a fence in a list item of a text of a document', () => { - it('ends with the item, at the first line indented less than its content', () => { - expect([ - findingsOf('Text', ' ```', '## Details'), - findingsOf('- item', ' ```', '## Details'), - findingsOf('- item', ' ```', '## Details', ' ```'), - findingsOf('- item', ' ```', '## After it'), - findingsOf('- item', '', ' ```', ' # Inside', ' ```', '# After it'), - ]).toEqual([ - [[2, 'details']], - [[2, 'details']], - [ - [2, 'details'], - [3, 'fence'], - ], - [[2, 'heading']], - [[5, 'heading']], - ]); - }); - - it('is in no list item that starts empty when a blank line follows its marker, which ends the item', () => { - expect([ - findingsOf('1.', '', ' ```', ' ```', '# After it'), - findingsOf('-', ' ```', ' # Inside', ' ```', '# After it'), - ]).toEqual([[[4, 'heading']], [[4, 'heading']]]); - }); -}); - -describe('a block a text of a document leaves open', () => { - it('is refused at the line that opens it, a fence of backticks or of tildes', () => { - expect([findingsOf('Intro.', '```ts', 'code'), findingsOf('~~~', 'code', '```')]).toEqual([ - [[1, 'fence']], - [[0, 'fence']], - ]); - }); - - it('is refused at the line that opens it, an HTML block that ends at a closing tag of its raw kind', () => { - expect(['
', '