diff --git a/CLAUDE.md b/CLAUDE.md index 2f9941d..a1d6879 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -18,8 +18,8 @@ For work on the world package or the corpus (the disembodied simulation), read ` 6. `docs/specs/WORDFORM_PIPELINE.md` — the word-form pipeline: a Python program in `python/semantic_world/wordforms/` that makes spoken word forms, synthesizes them, and builds sound embeddings. 7. `docs/specs/CORPUS_GENERATOR.md` — the corpus generator: a Python program in `python/semantic_world/corpus/` that writes documents about the taxonomy's world in an artificial language, with a parse tree and a logical form for every sentence. Its numbered decisions, and the proposals they point to, are part of the specification. 8. `docs/specs/CONNECTED_SPEECH.md` — the plan for spoken sentences: whole utterances, alignment, pauses, speakers, and register. Planned, not built. -9. `docs/specs/WORLD_AND_LANGUAGE.md` — the world-and-language refactor: one world model of state and change for both simulation modes (fluents, event types with preconditions and effects, time steps), a new package `python/semantic_world/world/`, and the corpus rebuilt on that model. Built in phases and stages. Its decisions were reviewed by Jon on October 7, 2026, and are logged as WM.1 to WM.33 in `docs/DECISIONS.md`. Phase (a) is built (stages a1 to a8: the world package, the cut-over of the taxonomy and the corpus, the labels, states, changes, causal statements, descriptions, the documentation, and the regenerated datasets); the engineering choices of each stage are in `docs/proposals/` and logged as WM.E rows. Phase (b) is built: stage b1 (conditional effects), stage b2 (the world's comparisons and numeric fluents), stage b3a (the corpus's comparisons of scalars: relative gradable adjectives as comparisons, comparatives, and the comparison classes read from the world), and stage b3b (the numeric fluents in the corpus, `RAISES` and `LOWERS`, conditional causal statements, and the tense affix on the auxiliaries), each with a proposal file in `docs/proposals/`. Phase (c) is specified in `docs/specs/WORLD_PHASE_C.md` (item 10) and has begun: stage c1 (places, the static side) is built, after Jon's ruling WM.60 that every real the world draws is rounded to 4 decimals when drawn. -10. `docs/specs/WORLD_PHASE_C.md` — phase (c) of the world-and-language refactor: pair fluents (two-place fluents with profiles, `PAIRFL.`), places from a second run of the taxonomy generator, location as a pair fluent toward places, place event types (moves and acts at a place), emergent habitats, and the corpus's statements about them. Decided with Jon on October 15 and 16, 2026 (WM.36 to WM.59). Built in seven stages, c1 to c5b; stage c1 (places: the taxonomy's `labels.kind`, the second taxonomy run, place event types with requirements only, their capacities and statistics, and `view --places`) is built, and c2a is next; read `WORLD_AND_LANGUAGE.md` first. +9. `docs/specs/WORLD_AND_LANGUAGE.md` — the world-and-language refactor: one world model of state and change for both simulation modes (fluents, event types with preconditions and effects, time steps), a new package `python/semantic_world/world/`, and the corpus rebuilt on that model. Built in phases and stages. Its decisions were reviewed by Jon on October 7, 2026, and are logged as WM.1 to WM.33 in `docs/DECISIONS.md`. Phase (a) is built (stages a1 to a8: the world package, the cut-over of the taxonomy and the corpus, the labels, states, changes, causal statements, descriptions, the documentation, and the regenerated datasets); the engineering choices of each stage are in `docs/proposals/` and logged as WM.E rows. Phase (b) is built: stage b1 (conditional effects), stage b2 (the world's comparisons and numeric fluents), stage b3a (the corpus's comparisons of scalars: relative gradable adjectives as comparisons, comparatives, and the comparison classes read from the world), and stage b3b (the numeric fluents in the corpus, `RAISES` and `LOWERS`, conditional causal statements, and the tense affix on the auxiliaries), each with a proposal file in `docs/proposals/`. Phase (c) is specified in `docs/specs/WORLD_PHASE_C.md` (item 10) and has begun: stage c1 (places, the static side) is built, after Jon's ruling WM.60 that every real the world draws is rounded to 4 decimals when drawn, and stage c2a (pair fluents from an event file: the definition, the runtime, and the fixtures) is built, after Jon's ruling WM.61 that the taxonomy's similarity range is written at 6 decimals. +10. `docs/specs/WORLD_PHASE_C.md` — phase (c) of the world-and-language refactor: pair fluents (two-place fluents with profiles, `PAIRFL.`), places from a second run of the taxonomy generator, location as a pair fluent toward places, place event types (moves and acts at a place), emergent habitats, and the corpus's statements about them. Decided with Jon on October 15 and 16, 2026 (WM.36 to WM.59). Built in seven stages, c1 to c5b; stage c1 (places: the taxonomy's `labels.kind`, the second taxonomy run, place event types with requirements only, their capacities and statistics, and `view --places`) and stage c2a (pair fluents between entities as an event file defines them: their symbols, literals, initial pairs, compiled literals, the runtime's state and invariant, non-interference over pairs, the `spatial` block, pair changes in histories, and the fixtures) are built, and c2b (the generation of pair fluents) is next; read `WORLD_AND_LANGUAGE.md` first. 11. `docs/DECISIONS.md` — the log of every design decision of the taxonomy, word-form, and corpus programs, and of the world model. Append new decisions to it. 12. `docs/guides/` — user guides for the world package (`WORLD.md`: what a world is, the commands, the outputs, the configuration, the default world's figures, the two-place share and its levers), the taxonomy generator (`TAXONOMY.md`), the word-form pipeline (`WORDFORMS.md`), and the corpus generator (`CORPUS.md`). `README.md` there has the full chain. 13. `docs/ENVIRONMENT_SURVEY.md` — background only: why the stack was chosen. @@ -106,6 +106,7 @@ python -m semantic_world.corpus render runs/corpus/tiny_seed1 --wordforms runs/w python -m semantic_world.wordforms all data/wordforms/corpus_default.yaml # the word forms of the default corpus (after generate data/corpus/default.yaml); "forms" in place of "all" makes the word forms without audio pytest tests/corpus # the corpus generator's tests alone python -m semantic_world.world define data/world/default.yaml [--seed N] [--out DIR] # generate a world: the embedded taxonomy run, definition.json, entities.csv, derived values, statistics +python -m semantic_world.world define data/world/tiny_pairs.yaml # the tiny world with the pair fluents of events/pair_example.yaml (stage c2a): initial_pairs.csv beside entities.csv python -m semantic_world.world view runs/world/tiny_seed1 --preset classic [--include PROPERTY,PART] [--out FILE] # a table for a model: base, static, or classic python -m semantic_world.world view runs/world/tiny_seed1 --places [--preset static] # a table of the places (stage c1): base, static (with the determined place features and CANBE capacities), or classic python -m semantic_world.world simulate runs/world/tiny_seed1 --episodes 100 [--seed N] [--legal] [--out FILE] [--config data/corpus/default.yaml] # run episodes of a world run and write episodes.jsonl; --config reads a corpus file's scene block diff --git a/data/world/events/pair_example.yaml b/data/world/events/pair_example.yaml new file mode 100644 index 0000000..c0efa12 --- /dev/null +++ b/data/world/events/pair_example.yaml @@ -0,0 +1,49 @@ +# A worked example of an event file with pair fluents (docs/specs/WORLD_PHASE_C.md, "Explicit +# event files"; stage c2a). Over abstract labels, it defines four pair fluents between entities, +# one of each symmetry and two of them functional, and five event types that read and write them: +# EVENTTYPE2.1.1 ("approach") makes the agent near the patient, a symmetric add that writes both +# directions; EVENTTYPE2.1.2 ("follow") needs nearness and makes the patient the agent's one +# followed partner, a functional add whose compiled literal NOT SOME.PAIRFL.3(agent) keeps the +# profile, and marks the patient as not awake when the patient has something above it; +# EVENTTYPE2.2.1 ("mount") needs the two to follow the same partner and the agent awake, and puts +# the agent above the patient in the asymmetric functional PAIRFL.4, whose compiled literals are +# NOT PAIRFL.4(patient, agent) and NOT SOME.PAIRFL.4(agent); EVENTTYPE2.2.2 ("leave") needs +# nearness and something below the agent in PAIRFL.2, deletes the nearness (both directions), +# and sets a fluent of the patient when the patient is above the agent; EVENTTYPE1.1 ("rest") +# needs the agent to follow someone. The requirements are explicit and always hold, so every +# ordered pair is able and legality is the preconditions' alone. Event types not listed are +# sampled, and so are their requirements. Use it with the tiny world: data/world/tiny_pairs.yaml +# names it as event_types.event_file, relative to the configuration file's folder. + +pair_fluents: + PAIRFL.1: {symmetry: symmetric, functional: false, initial_rate: 0.2} # say, near + PAIRFL.2: {symmetry: asymmetric, functional: false, initial_rate: 0.1} # say, above (initial pairs only) + PAIRFL.3: {symmetry: general, functional: true, initial_rate: 0.2} # say, follows + PAIRFL.4: {symmetry: asymmetric, functional: true, initial_rate: 0.0} # say, carries (no initial pairs) +fluents: + BOOLFL.1: {initial_rate: 0.8} # say, awake +event_types: + EVENTTYPE2.1.1: # approach + requirement: "TRUE" + precondition: "NOT PAIRFL.1(agent, patient)" + effects: ["PAIRFL.1(agent, patient) := 1"] + EVENTTYPE2.1.2: # follow + requirement: "TRUE" + precondition: "PAIRFL.1(agent, patient)" + effects: + - "PAIRFL.3(agent, patient) := 1" + - "when SOME.PAIRFL.2(patient) then patient.BOOLFL.1 := 0" + EVENTTYPE2.2.1: # mount + requirement: "TRUE" + precondition: "SAME.PAIRFL.3(agent, patient) AND agent.BOOLFL.1" + effects: ["PAIRFL.4(agent, patient) := 1"] + EVENTTYPE2.2.2: # leave + requirement: "TRUE" + precondition: "PAIRFL.1(agent, patient) AND SOMEOF.PAIRFL.2(agent)" + effects: + - "PAIRFL.1(agent, patient) := 0" + - "when PAIRFL.2(patient, agent) then patient.BOOLFL.2 := 1" + EVENTTYPE1.1: # rest + requirement: "TRUE" + precondition: "SOME.PAIRFL.3(agent)" + effects: ["agent.BOOLFL.3 := 1"] diff --git a/data/world/tiny_pairs.yaml b/data/world/tiny_pairs.yaml new file mode 100644 index 0000000..84e8764 --- /dev/null +++ b/data/world/tiny_pairs.yaml @@ -0,0 +1,17 @@ +# The tiny world with the pair example's pair fluents and explicit event types (stage c2a). + +name: tiny_pairs +seed: 1 +taxonomy: {config: data/taxonomy/tiny_relations.yaml} +places: {taxonomy: {config: data/taxonomy/places_tiny.yaml}} +place_event_types: {count: 2} + +fluents: + count: 4 + +event_types: + unary: {count: 4} + binary: + features: {count: 4, expected_true: 2} + taxonomy: {superordinates: 2, depth: 2, branching: 2} + event_file: events/pair_example.yaml diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index c79c11c..da3d3f5 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -407,6 +407,8 @@ Spec: `docs/specs/WORLD_AND_LANGUAGE.md`. | WM.58 | 2026-10-16 | Where a preposition stands when its object is the gap of a relative clause is a setting, `grammar.word_order.stranding: strand | pied_pipe`, default `strand`. | Claude (chat) | Jon | Decided | `specs/WORLD_PHASE_C.md`, decision 23 | | WM.59 | 2026-10-16 | In phase (c), a conditional effect with a `pair` condition makes no causal statement; the grammar it needs joins stage b3b's second relative clause on the language to-do list. | Claude (chat) | Jon | Decided | `specs/WORLD_PHASE_C.md`, decision 24 | | WM.60 | 2026-10-16 | Every real the world draws and stores at full precision is rounded to 4 decimals when drawn, as thresholds and comparison margins are (REL.9): a numeric fluent's drift, each entity's initial value of a numeric fluent (uniform or beta, when the world is generated and when an episode redraws its initial state), and a numeric effect's constant. Reason: on Linux (x86_64), with the same Python and package versions as the Mac, the tiny world's definition differed from the Mac's in the last digit of drawn reals (an amount of 0.3969666988843933 against 0.39696669888439323, a drift of 0.005553129001236876 against 0.005553129001236879), so the rule-set identity differed between machines and the snapshot's identity tests failed; the likely cause is that `rng.uniform` computes `low + (high - low) * u`, which ARM and x86 can round differently (a fused multiply-add). The rounding changes the tiny and default worlds once (stage c1, item 1). | Claude (chat) | Jon | Decided | Jon's ruling of October 16, 2026; stage c1 proposal, item 1 | +| WM.61 | 2026-10-21 | The taxonomy's similarity range, `superordinates.similarity_min` and `similarity_max` of `summary.yaml`, is rounded to 6 decimals when the file is written, as the CSVs are; the value the similarity bound is checked against stays unrounded, and no draw changes. Reason: on Linux (x86_64) with the same Python and package versions as the Mac, every output of every world and corpus matched the Mac's, and every taxonomy output but `summary.yaml` of the configurations whose superordinates have a similarity bound, where the two values differed in the 14th digit (-0.07165743639000198 against -0.07165743639000184): they come from a matrix product, which the two machines' math libraries compute differently. The golden hashes of `summary.yaml` moved once (stage c2a, item 1). Numeric threshold points stay unrounded, because plain arithmetic gives the same result on every machine. | Claude (chat) | Jon | Decided | Jon's ruling of October 21, 2026, on stage c1's open questions 1 and 2; stage c2a proposal | +| WM.62 | 2026-10-21 | In stage c3, `data/world/tiny.yaml`'s `place_event_types.constraint_families` is changed so that every tiny place requirement reads the place (for example `{patient: 1, cross: 1, key_lock: 1}`, with the agent family at 0), because c3's tests of habitats and of acts at a place need tiny place requirements that read the place. Stage c1's seed-1 tiny place event types read the agent alone (c1's open question 4). The place constraints keep borrowing the key-lock and comparison settings of `event_types.binary` until a world needs them apart (c1's question 3), and the level rule of the relation proportions (WM.E251) is accepted as built (c1's question 5). | Claude (chat) | Jon | Decided | Jon's ruling of October 21, 2026, on stage c1's open questions 3 to 5; stage c2a proposal; built in stage c3 | ### World model: Claude Code's engineering choices @@ -669,6 +671,20 @@ Claude Code makes these while building, and records them in each stage's proposa | WM.E253 | 2026-10-16 | The fixture format gains `places` (the rows of `places.csv`) after `entities`, written only for a world with places; `check_fixture` and the brute-force evaluator read it (`BruteForce(record, entities, places)`, with `_row` finding an entity or a place by label), a place event type's bindings in the evaluator are every entity paired with every place, and the regenerated tiny fixtures carry the places and the place scopes while listing the legal bindings of the performable event types only. The hand-written fixtures are unchanged. | Claude Code | Jon | Working design | Stage c1 proposal, choice 14 | | WM.E254 | 2026-10-16 | `view --places` writes `RUN/views/places_.csv`: `base` (label, leaf, free place features, place scalars), `static` (with the determined place features and the `CANBE.PLACEEVENTTYPE` capacities), and `classic` (with the ISA columns of the place tree); the sidecar records `places: true`, and `--include` works over the place columns. | Claude Code | Jon | Working design | Stage c1 proposal, choice 15 | | WM.E255 | 2026-10-16 | Checked, not changed: with places on, the tiny corpus's `documents.jsonl`, `lexicon.csv`, `stats.yaml`, and every rendering are byte-identical to the corpus over the same world with `places: null`, and `scenes.jsonl` differs in the rule-set identity alone; no corpus output names a place, a place feature, or a place event type. | Claude Code | Jon | Working design | Stage c1 proposal, choice 16 | +| WM.E256 | 2026-10-21 | `summary_stats` in the taxonomy's `analysis.py` rounds the similarity range to 6 decimals when it builds the summary (WM.61); a null range stays null, and the bound is checked against the unrounded value. The golden hashes of `summary.yaml` moved for `default`, `relations`, `rule_file`, `tiny`, and `tiny_relations` (the two tiny taxonomies have a similarity bound too); the a5a comparison test rounds the golden folders' values, which are never edited. A scan of every YAML and JSON output of the reference runs found no other real computed by a matrix product or a library function and written at full precision: beside the range, only `mean_true_features_per_instance`, an exact mean of integers. | Claude Code | Jon | Working design | Stage c2a proposal, choice 1 | +| WM.E257 | 2026-10-21 | Two streams after `world:place_event_types`: `world:pair_fluents` (from which stage c2b draws the profiles and initial rates; nothing is drawn from it in c2a) and `world:pair_initial` (the initial pairs, fluent by fluent in label order). A world without an event file that defines pair fluents draws nothing from either, so its definition, identity, and every file but `config.yaml` (which lists the two seeds) are those of stage c1. | Claude Code | Jon | Working design | Stage c2a proposal, choice 2 | +| WM.E258 | 2026-10-21 | `pair_fluents.py`: `PairFluent` (label, index, symmetry, functional, initial rate, `total` always False between entities) and `PairFluents` (the fluents and their initial pairs, one Boolean matrix per fluent with the entities as rows and the fluent's targets as columns, so that stage c3's location fluents add a second target kind). The event file's `pair_fluents` block must label its entries `PAIRFL.1` to `PAIRFL.k` without a gap; `total: true` is refused until c3, and so are a `place_event_types` entry and a `locations` key. The initial pairs follow the specification's procedure in a fixed draw order: for a functional pair fluent one draw against the rate per entity in entity order and then the partner uniformly among the entities that keep the profile (an entity whose row is already filled, or with no eligible partner, gets none); otherwise one draw per unordered pair (`i < j`, row by row) for a symmetric or an asymmetric pair fluent, with one more draw for the direction of an asymmetric pair, and one draw per ordered pair for a general one; a rate of 0 draws as the others do and selects nothing. `initial_pairs.csv` has the columns `pair_fluent`, `first`, `second`, one row per ordered pair that holds, both directions of a symmetric pair, by pair fluent, then first and second in entity order. | Claude Code | Jon | Working design | Stage c2a proposal, choice 3 | +| WM.E259 | 2026-10-21 | `definition.json` gains, only with pair fluents: the symbol kind `pair_fluent` (`label`, `kind`, `derived` false, `fluent` true, `arity` 2, `symmetry`, `functional`, `total`, `initial_rate`, all hashed), after the numeric fluents and before the event types; the entity-scope literals `pair_some` and `pair_someof` (`role` null, `pair_fluent`; keys `SOME.PAIRFL.k` and `SOMEOF.PAIRFL.k`) in the literal table where a derived fluent's rule reads them, after the numeric thresholds, computed before the matrices from the state's pairs; and, in the event-type records, pair literals of the four kinds (`kind`, `pair_fluent`, `first` and `second` or `role`, `value`, and `origin` only for a compiled literal) in preconditions and conditions, and pair effects (`kind: pair`, `pair_fluent`, `first`, `second`, `value`, `expression`). The loader accepts all four kinds in the literal table (the two-argument kinds with the role `binding`), though the generator writes the entity-scope kinds only, because preconditions and conditions carry their pair literals in the records as the Boolean and numeric literals do. | Claude Code | Jon | Working design | Stage c2a proposal, choice 4 | +| WM.E260 | 2026-10-21 | `dynamics.py` gains `PairLiteral` (kind, pair fluent, first, second, value, origin; `role` and `fluent` name the first role and the pair fluent so that code that counts literals by role reads it as the other kinds) and `Effect.second` (the second role of a pair effect, `Effect.pair`). The event-file parser replaces each pair literal's text with a placeholder name before the taxonomy's expression parser runs, because the parentheses and the comma of `PAIRFL.1(agent, patient)` are not in the expression grammar, and maps the placeholders back; `parse_condition` recognizes pair literals by their form. | Claude Code | Jon | Working design | Stage c2a proposal, choice 5 | +| WM.E261 | 2026-10-21 | What a pair literal names, for the rules of "Preconditions", "Effects", and "Conditions (F5)": a pair (both directions of a symmetric pair fluent are one pair), the row of a role (`SOME`; `SOMEOF` of a symmetric pair fluent names the same set), the column of a role (`SOMEOF`), or the rows of both roles (`SAME`); two literals that name the same thing with different values contradict, two effects on one pair with different values contradict, an add whose precondition requires the pair or a delete whose precondition requires its absence changes nothing, and a pair condition literal names no pair or row the precondition names, nor the pair the effect itself writes (as a Boolean condition never names the fluent the effect sets). Achievable: a positive literal when its pair fluent's initial rate is above 0 or some effect adds a pair of it; a negative one when the rate is below 1, the pair fluent is functional, or some effect deletes one; `SAME` on a symmetric functional pair fluent is never achievable, because two entities with one partner would give that partner two. Each rule is a `ConfigError` naming the file and the entry for an explicit entry, and a `ValueError` of `check_dynamics` naming the event type for any definition. | Claude Code | Jon | Working design | Stage c2a proposal, choice 6 | +| WM.E262 | 2026-10-21 | The compiled literals (F4) are appended to every event type with an unconditional add effect after the fix-ups and before the conditions are drawn, in effect order (`NOT PAIRFL.k(second, first)` for an asymmetric pair fluent, `NOT SOME.PAIRFL.k(first)` for a functional one, and `NOT SOME.PAIRFL.k(second)` too for a symmetric functional one), with the origin `profile`, a literal already present by what it names not added twice; an explicit entry whose compiled literals contradict its precondition is an error naming the file and the entry, and a pair condition is checked against the compiled literals too. `redraw_preconditions` keeps the compiled literals, and `check_dynamics` requires every add to carry them. A conditional add on an asymmetric or functional pair fluent is refused. | Claude Code | Jon | Working design | Stage c2a proposal, choice 7 | +| WM.E263 | 2026-10-21 | The runtime: `State.pairs` is a tuple of Boolean matrices, one per pair fluent in symbol order, read-only, irreflexive; a definition that carries no initial pairs (a fixture's) starts with none. `derive` fills the `pair_some` and `pair_someof` literals from the state's pairs before the entity layers; `pair_literal_holds` judges the four kinds over a bindings array (a symmetric pair fluent holds both directions, so a literal about it reads both). The facts an event reads and writes are keyed `(entity, fluent)` for a base fluent and `(pair fluent, first, second)` for an ordered pair; a pair effect writes its pair and, for a symmetric pair fluent, the reverse; a literal reads its pair (both directions when symmetric), every pair of the row (`SOME`), of the column (`SOMEOF`), or of both rows (`SAME`), rows and columns ranging over every entity of the definition, and a literal on a derived fluent reads, through `pair_cone`, the rows and columns of the entity its rule reads. After the step every profile is checked, and a violation is `ProfileViolationError` (kind `profile`), naming the pair fluent, the pairs, and the events; no fixture ends in it. | Claude Code | Jon | Working design | Stage c2a proposal, choice 8 | +| WM.E264 | 2026-10-21 | The `spatial` block (F6): the event file's block is checked for its form (a pair fluent of the file, a `detector` name, a `parameters` mapping of numbers or strings) and written last in `definition.json` as `spatial` with `spatial_id`, the SHA-256 of its canonical JSON, both absent without a block and neither hashed; the loader checks the form and the identity and keeps both on `RuntimeDefinition`. | Claude Code | Jon | Working design | Stage c2a proposal, choice 9 | +| WM.E265 | 2026-10-21 | Episodes and histories: an episode's initial state holds the initial pairs among its participants and no other pair (WM.38), so a history's `initial_pairs` (after `initial_numeric`; every ordered pair that holds, both directions of a symmetric pair, as `[pair fluent, first, second]` triples) is the whole pair part of its initial state, and `replay` builds the state from it; under `scene.initial: redraw` the pairs are drawn again among the participants, in entity order, by the initial-pair procedure at each pair fluent's rate, after the Boolean and numeric redraws, so episodes of a world without pair fluents are unchanged. A pair change is `{"pair", "first", "second", "to"}`, one per pair effect whose fired write changed the value, in the direction the effect names; `replay` checks every event's pair changes against `pair_changes_of`. A world without pair fluents writes neither key. | Claude Code | Jon | Working design | Stage c2a proposal, choice 10 | +| WM.E266 | 2026-10-21 | The brute-force evaluator reads the pair part of a state as sets of label pairs (`BrutePairs`), never through the matrices: the pair literals of rules, preconditions, and conditions, the pair effects (both directions of a symmetric pair), the read and write sets, and the profiles after a step. The fixture format gains `initial_pairs` after `initial_numeric` and `pairs` after `numeric` in each step, both lists of triples in the order of `initial_pairs.csv`; the version stays 1. The hand world gains three pair fluents (symmetric, asymmetric, symmetric functional), a derived fluent `BOOLFL.4 = SOME.PAIRFL.1`, and five event types (a symmetric add, a symmetric delete, an asymmetric add and a functional add with their compiled literals, and a pair condition); `hand_29` to `hand_37` are fixtures 1 to 4, 10, 12 to 14, and 16 of the specification, and the earlier hand fixtures gained `initial_pairs: []`, `pairs: []`, and the new event types' legal bindings; the tiny fixtures did not change. | Claude Code | Jon | Working design | Stage c2a proposal, choice 11 | +| WM.E267 | 2026-10-21 | `load_world` of the corpus refuses a world with pair fluents, naming stage c4; `world_stats.yaml` counts a pair condition literal under `by_symbol.pair` only when the world has one; `absorbing_fluents` and `enabling_edges` leave pair effects out and `episode_stats` counts Boolean changes only (the pair statistics are stage c2b's); the `define` line names the pair fluents only when there are any. Every other statistic and figure of a world without pair fluents is unchanged. | Claude Code | Jon | Working design | Stage c2a proposal, choice 12 | +| WM.E268 | 2026-10-21 | `data/world/events/pair_example.yaml` and `data/world/tiny_pairs.yaml`: four pair fluents (symmetric, asymmetric, general functional, asymmetric functional with the rate 0), five explicit event types over them with every kind of pair literal, adds and deletes, and two pair conditions, each with the explicit requirement `TRUE`, which keeps the constraints of the features defining at the parent (WM.E106), so that every event type occurs in the statistics episodes. | Claude Code | Jon | Working design | Stage c2a proposal, choice 13 | +| WM.E269 | 2026-10-21 | `rounded_similarity` in the taxonomy's `analysis.py` rounds the similarity range to 6 decimals and adds `0.0`, so a rounded zero is written as `0.0`, never `-0.0` (Jon's ruling of October 21, 2026, extending WM.61: the tiny taxonomies' range is about -1.24e-17, noise whose sign the rounding was meant to remove). The golden hashes of `summary.yaml` moved for `tiny` and `tiny_relations`. Every other value the world or the taxonomy rounds when drawn or written (the drift, the numeric effect constants, the initial numeric values, the scalar thresholds, the comparison margins, and the rounded statistics) could in principle give `-0.0`, but a scan of every output of every configuration in `data/` found none, so the rule is applied to the similarity range alone. | Claude Code | Jon | Working design | Stage c2a proposal, choice 14 | ## Process diff --git a/docs/guides/CORPUS.md b/docs/guides/CORPUS.md index e879676..d2f8a68 100644 --- a/docs/guides/CORPUS.md +++ b/docs/guides/CORPUS.md @@ -32,7 +32,7 @@ The default corpus is 10,000 documents about the default world, `data/world/defa PYTHONPATH=python python -m semantic_world.corpus generate data/corpus/default.yaml ``` -It prints `10000 documents, 100918 sentences, 483322 tokens, 10082 scenes, 46 test sets with 16457 pairs`, and takes about 2.2 minutes (133 seconds) on a laptop (October 15, 2026, stage b3a; 158 seconds in stage b2, 157 seconds in stage b1, 158 seconds in stage a8). The world itself takes 6 seconds (`WORLD.md`). Every stage records these times, measured with nothing else running, and flags a growth of more than half in its proposal file; there is no hard time limit, and no step is made cheaper by changing what it produces by default (Jon's ruling 3 on stage a8). +It prints `10000 documents, 100918 sentences, 483322 tokens, 10082 scenes, 46 test sets with 16457 pairs`, and takes about 2.4 minutes (142 seconds) on a laptop (October 21, 2026, stage c2a; 134 seconds in stage c1, 141 seconds in stage b3b, 133 seconds in stage b3a, 158 seconds in stage b2, 157 seconds in stage b1, 158 seconds in stage a8). The world itself takes 7 seconds (`WORLD.md`). Every stage records these times, measured with nothing else running, and flags a growth of more than half in its proposal file; there is no hard time limit, and no step is made cheaper by changing what it produces by default (Jon's ruling 3 on stage a8). Two options change a run without editing the configuration: diff --git a/docs/guides/README.md b/docs/guides/README.md index ba2b15d..c4fed48 100644 --- a/docs/guides/README.md +++ b/docs/guides/README.md @@ -8,7 +8,7 @@ The specifications in `docs/specs/` are a different kind of document. Specificat | Guide | Program | Status | | --- | --- | --- | -| [`WORLD.md`](WORLD.md) | The world package: entities, fluents (Boolean and numeric), event types with requirements, preconditions, and effects, comparisons, places and place event types, episodes, histories, and views | Phases (a) and (b) complete; phase (c) begun (stage c1, places) | +| [`WORLD.md`](WORLD.md) | The world package: entities, fluents (Boolean and numeric), event types with requirements, preconditions, and effects, comparisons, places and place event types, episodes, histories, and views | Phases (a) and (b) complete; phase (c) begun (stage c1, places; stage c2a, pair fluents from an event file) | | [`TAXONOMY.md`](TAXONOMY.md) | The taxonomy generator: categories, instances, features, rules, and scalar dimensions (the part of a world that is fixed), for entities or for places | Complete | | [`CORPUS.md`](CORPUS.md) | The corpus generator: documents in an artificial language about a world, their renderings, test sets, and spoken word forms | Complete | | [`WORDFORMS.md`](WORDFORMS.md) | The word-form pipeline: spoken word forms, synthesis, auditory front ends, and sound embeddings | Stages 1–7 complete | @@ -33,14 +33,14 @@ python -m semantic_world.corpus render runs/corpus/default_seed1 --wordforms run | Step | Output folder | What the step writes | Time at default scale | | --- | --- | --- | --- | -| `define` | `runs/world/default_seed1/` | The embedded taxonomy run, the place taxonomy run, the definition, the entities and the places, the derived values, and the world's statistics | 6 seconds (6.4 s in stage c1, 6.3 s in stage b3b, 6.1 s in stage b3a, 6.7 s in stage b2, 7.6 s in stage b1, 7.8 s in stage a8) | -| `generate` | `runs/corpus/default_seed1/` | Documents, the four renderings, test sets, statistics, and the word-form request | 2.2 minutes (134 s in stage c1, 141 s in stage b3b, 133 s in stage b3a, 158 s in stage b2, 157 s in stage b1, 158 s in stage a8) | +| `define` | `runs/world/default_seed1/` | The embedded taxonomy run, the place taxonomy run, the definition, the entities and the places, the derived values, and the world's statistics | 7 seconds (6.8 s in stage c2a, 6.4 s in stage c1, 6.3 s in stage b3b, 6.1 s in stage b3a, 6.7 s in stage b2, 7.6 s in stage b1, 7.8 s in stage a8) | +| `generate` | `runs/corpus/default_seed1/` | Documents, the four renderings, test sets, statistics, and the word-form request | 2.4 minutes (142 s in stage c2a, 134 s in stage c1, 141 s in stage b3b, 133 s in stage b3a, 158 s in stage b2, 157 s in stage b1, 158 s in stage a8) | | `wordforms all` | `runs/wordforms/corpus_default_seed1/` | Word forms, audio, auditory front ends, sound embeddings, and their evaluation | about 30 minutes | | `render` | `runs/corpus/default_seed1/` | The spelled rendering (`corpus.txt`) and the word columns of `lexicon.csv` | 3 seconds | Every stage records these times, measured with nothing else running, and its proposal file flags a growth of more than half, with its cause. There is no hard time limit, and no step is made cheaper by changing what it produces by default: a step that becomes too slow gets a setting that a study can turn on, with the full version as the default (Jon's ruling 3 on stage a8). -The default chain ran on October 16, 2026 (stage c1: the world and the corpus regenerated, and the corpus rendered with the word forms of stage b3b, whose request did not change; the word forms themselves last ran on October 15, 2026, stage b3b): a world of 284 entities, 8 fluents, 2 numeric fluents, and 40 event types with 10 conditional and 24 numeric effects, and 12 places with 4 place event types; 10,000 documents with 100,918 sentences; spoken words for 203 content lexemes and 18 function words with five sound embeddings; and 16,457 pairs of test items in 46 sets. The tiny configurations (`data/world/tiny.yaml`, `data/corpus/tiny.yaml`, and `data/wordforms/corpus_tiny.yaml`) run the same chain in under a minute. With `forms` in place of `all`, the word-form step skips the audio, which `render` does not need. +The default chain ran on October 21, 2026 (stage c2a: the world and the corpus regenerated, with the same definition, identity, and corpus as stage c1; the corpus was last rendered on October 16, 2026, stage c1, with the word forms of stage b3b, whose request did not change; the word forms themselves last ran on October 15, 2026, stage b3b): a world of 284 entities, 8 fluents, 2 numeric fluents, and 40 event types with 10 conditional and 24 numeric effects, and 12 places with 4 place event types; 10,000 documents with 100,918 sentences; spoken words for 203 content lexemes and 18 function words with five sound embeddings; and 16,457 pairs of test items in 46 sets. The tiny configurations (`data/world/tiny.yaml`, `data/corpus/tiny.yaml`, and `data/wordforms/corpus_tiny.yaml`) run the same chain in under a minute. With `forms` in place of `all`, the word-form step skips the audio, which `render` does not need. The taxonomy generator also runs on its own, when only the world's categories and feature vectors are needed (`TAXONOMY.md`), and the world package runs episodes of a world without a corpus (`WORLD.md`, "simulate"). diff --git a/docs/guides/TAXONOMY.md b/docs/guides/TAXONOMY.md index ad50314..c9164d8 100644 --- a/docs/guides/TAXONOMY.md +++ b/docs/guides/TAXONOMY.md @@ -200,7 +200,7 @@ One row per feature: the proportion of instances with the feature, its entropy, ### `summary.yaml` and `config.yaml` -`summary.yaml` gives counts, the mean number of true features per instance, constant features, duplicate leaves and instances, and any warnings. `config.yaml` records the complete resolved configuration and every random seed. Running the generator on an output folder's `config.yaml` reproduces the run exactly. +`summary.yaml` gives counts, the mean number of true features per instance, constant features, duplicate leaves and instances, the superordinates' similarity range (`similarity_min` and `similarity_max`, written at 6 decimals since stage c2a of the world-and-language refactor, as the CSVs are, because the range comes from a matrix product that differs between machines in the 14th digit; Jon's ruling WM.61), and any warnings. `config.yaml` records the complete resolved configuration and every random seed. Running the generator on an output folder's `config.yaml` reproduces the run exactly. ### `rule_matrices.json` diff --git a/docs/guides/WORLD.md b/docs/guides/WORLD.md index 0d18b89..9910ae3 100644 --- a/docs/guides/WORLD.md +++ b/docs/guides/WORLD.md @@ -2,9 +2,9 @@ The world package defines a world and runs it. A world is a set of entities with fixed features, a set of fluents that change over time, and a set of event types that say what can happen to the entities and what each event changes. The package generates a world from a taxonomy of the taxonomy generator, writes the world's definition as data files, runs episodes of the world, and writes their histories. The corpus generator reads worlds and writes documents about them. The 3D engine will later read the same definition files. -This guide covers running the package, what a world is, the commands, the output files, the configuration file, the default world's figures, the conformance fixtures, the two-place share, and the Python interface. The design is specified in `docs/specs/WORLD_AND_LANGUAGE.md` and, for phase (c), `docs/specs/WORLD_PHASE_C.md`, and every design decision is listed in `docs/DECISIONS.md` (WM.1 to WM.60, and the engineering choices WM.E1 and following). +This guide covers running the package, what a world is, the commands, the output files, the configuration file, the default world's figures, the conformance fixtures, the two-place share, and the Python interface. The design is specified in `docs/specs/WORLD_AND_LANGUAGE.md` and, for phase (c), `docs/specs/WORLD_PHASE_C.md`, and every design decision is listed in `docs/DECISIONS.md` (WM.1 to WM.62, and the engineering choices WM.E1 and following). -**Status.** Phase (a) of the world-and-language refactor is complete (stages a1 to a8): Boolean fluents, one-place and two-place event types with requirements, preconditions, and effects, time steps with inertia, the Python runtime, episodes with selection policies, histories, views, world statistics, and the conformance fixtures. Phase (b) has begun: stage b1 adds conditional effects (an effect that applies only when a condition on the binding holds), and stage b2 adds the comparisons of scalars (`GREATER`, `ABOVE`, `BELOW`, with comparison classes and `ENTITY`) and numeric fluents (real-valued fluents with ranges, initial distributions, drift, numeric effects, numeric threshold literals in preconditions and conditions, and comparisons at a time point). Phase (b) is complete with stage b3 (b3a and b3b), which built the corpus's comparatives, numeric adjectives, and conditional causal statements (`CORPUS.md`). Phase (c) (`docs/specs/WORLD_PHASE_C.md`) has begun: stage c1 adds places, a second kind of object from a second taxonomy run, with place event types whose requirements read an entity's and a place's features, their capacities, comparison classes, and statistics; pair fluents (stages c2a and c2b), locations and the dynamics of place event types (c3), and the corpus's statements about them (c4 to c5b) are not built yet. Since stage c1 every real the world draws is rounded to 4 decimals when drawn (Jon's ruling WM.60), so the same seed gives the same world on every machine. Sorts and parts come in phase (d). The Rust runtime is not built yet: the fixtures are its tests, waiting. +**Status.** Phase (a) of the world-and-language refactor is complete (stages a1 to a8): Boolean fluents, one-place and two-place event types with requirements, preconditions, and effects, time steps with inertia, the Python runtime, episodes with selection policies, histories, views, world statistics, and the conformance fixtures. Phase (b) has begun: stage b1 adds conditional effects (an effect that applies only when a condition on the binding holds), and stage b2 adds the comparisons of scalars (`GREATER`, `ABOVE`, `BELOW`, with comparison classes and `ENTITY`) and numeric fluents (real-valued fluents with ranges, initial distributions, drift, numeric effects, numeric threshold literals in preconditions and conditions, and comparisons at a time point). Phase (b) is complete with stage b3 (b3a and b3b), which built the corpus's comparatives, numeric adjectives, and conditional causal statements (`CORPUS.md`). Phase (c) (`docs/specs/WORLD_PHASE_C.md`) has begun: stage c1 adds places, a second kind of object from a second taxonomy run, with place event types whose requirements read an entity's and a place's features, their capacities, comparison classes, and statistics; stage c2a adds pair fluents between entities as an event file defines them (two-place fluents with a profile, their literals in preconditions, conditions, and derived rules, pair effects, the compiled literals that keep the profiles, the runtime's invariant, non-interference over pairs, the `spatial` block, and pair changes in histories); the generation of pair fluents (c2b), locations and the dynamics of place event types (c3), and the corpus's statements about them (c4 to c5b) are not built yet. Since stage c1 every real the world draws is rounded to 4 decimals when drawn (Jon's ruling WM.60), so the same seed gives the same world on every machine. Sorts and parts come in phase (d). The Rust runtime is not built yet: the fixtures are its tests, waiting. ## Quick start @@ -54,6 +54,7 @@ A world definition has three parts: the entities' base facts, the feature rules, - **Time.** An episode has time points `TIME.1`, `TIME.2`, and so on. Step k takes the state at `TIME.k` to the state at `TIME.k+1`. Every event of a step must be legal at `TIME.k`, and the events of one step must not interfere: no two write the same fluent of the same entity (every effect counts as a write, whatever its condition; a numeric effect writes its numeric fluent), and none writes a fluent that another's precondition, or a condition of another's effect, reads (a condition on a derived fluent reads every base fluent the derived fluent depends on, numeric fluents read through thresholds included; a condition on a static feature reads nothing an event can write). The effects whose conditions hold are applied, every other base fluent keeps its value (inertia), every numeric fluent then moves by its drift, and the derived fluents are recomputed. Applying a step's events together gives the same state as applying them one at a time in any order, the drift added once after all of them. - **Episodes and histories.** An episode runs the world over a few participants for a few steps. Participants are drawn around a seed entity, weighted by thematic relatedness and taxonomic similarity. At each step a selection policy draws events among the legal ones; by default (`uniform_event`) it draws uniformly among all legal events, so an event type that is legal for many bindings happens more often, as in a world where some things simply happen more. An episode ends after its steps, or earlier when nothing is legal (quiescence). A **history** records the episode: its participants, its initial state (the numeric values among it), and each step's events with the changes they made, every numeric change with its cause (an event, or the drift). The corpus calls an episode a scene. - **Places** (stage c1 of phase (c)) are a second kind of object, the world's grain of interaction: `PLACE.1.2.1`, the instances of a second taxonomy run with labels of its own (`PLACECATEGORY`, `PLACEPROPERTY`, `PLACEPART`, `PLACESCALARDIM`), which the world runs in memory on the configuration `places.taxonomy` names. Places are not entities: they have place features and place scalars, no fluents, and no location, and nothing that reads the entities' tables sees a place. A **place event type** (`PLACEEVENTTYPE.2`) has an entity as its agent and a place as its patient; its requirement is a conjunction of one or two constraints over the agent's features and the place's features (the same families as the two-place constraints, the comparison family comparing an entity scalar with a place scalar), drawn again when it holds for no binding or for every binding of the world. Its capacities are `CAN.PLACEEVENTTYPE.2` (the entity is able to be the agent at some possible place) and `CANBE.PLACEEVENTTYPE.2` (the place is able to be the patient of some possible entity). In this stage a place event type has no precondition and no effects, and no episode includes one; stage c3 makes them moves and acts at a place, with location as a pair fluent toward places. `EVERYPLACE` is the comparison class of every place, as `ENTITY` is of every entity, and the runtime's comparisons accept places and means of categories of places on place scalars, never mixed with entity scalars. +- **Pair fluents** (stage c2a of phase (c)) are Boolean facts about two entities that change over time, `PAIRFL.1` to `PAIRFL.n`: near, above, follows. A pair fluent is stored as the set of ordered pairs for which it holds at a time point, and events add and delete its pairs. Each has a **profile**: its symmetry (`symmetric`: a pair holds exactly when its reverse does; `asymmetric`: never both; `general`: no constraint) and whether it is `functional` (at most one partner in each row, and for a symmetric functional pair fluent in each column too); every pair fluent is irreflexive, and a pair fluent between entities is never total (the total functional profile is location's, toward places, in stage c3). In this stage pair fluents come from an event file alone, with an **initial rate** each, at which the **initial pairs** are drawn once when the world is generated, respecting the profile (`initial_pairs.csv`); stage c2b draws them. The **pair literals** read them: `PAIRFL.2(agent, patient)` (the pair holds), `SOME.PAIRFL.2(agent)` (the agent's row holds a pair), `SOMEOF.PAIRFL.2(patient)` (the patient's column does), and `SAME.PAIRFL.3(agent, patient)` (the two share a partner, on a functional pair fluent), each positive or negated, in a precondition, in a condition, and (`SOME` and `SOMEOF` of the entity itself) in a derived fluent's rule. A **pair effect** adds or deletes the pair between the roles, `PAIRFL.2(agent, patient) := 1`; a symmetric add or delete writes both directions. The generator compiles each profile into precondition literals of the origin `profile` (F4): an add on an asymmetric pair fluent gains `NOT PAIRFL.k(patient, agent)`, an add on a functional one `NOT SOME.PAIRFL.k(agent)` (and `NOT SOME.PAIRFL.k(patient)` too when it is symmetric), so that a legal event never breaks a profile and two events that could jointly break one interfere; the runtime still checks every profile after each step (a `ProfileViolationError`, which a correct definition never raises). A conditional add on an asymmetric or functional pair fluent is not allowed, because its compiled literals would not be a plain precondition. For non-interference an event reads the pairs, rows, and columns its literals read (through the cones of derived fluents) and writes the pairs its effects write, a row or a column counting as every pair in it. An event file may also give a **`spatial` block**, a detector and parameters for a pair fluent, which the 3D runtime will read; `definition.json` carries it with `spatial_id`, outside the rule-set identity. The corpus does not state pair fluents yet (stage c4), and refuses a world that has them. - **Views** choose which columns of the base facts and the derived values a model sees. A view is an experiment setting, never part of the world. The event types attach their preconditions and effects to the tree as well: an event-type feature can carry a precondition literal or an effect, and every event type with that feature inherits it, so a branch of the tree shares what it needs and what it does, as it shares its constraints. @@ -76,12 +77,13 @@ Every label is a word a reader can read without a key, a period, and an index. T | Place property, place part, place scalar | `PLACEPROPERTY.`, `PLACEPART.`, `PLACESCALARDIM.` | `PLACEPROPERTY.3` | | Place event type, and its capacities | `PLACEEVENTTYPE.`, `CAN.PLACEEVENTTYPE.`, `CANBE.PLACEEVENTTYPE.` | `PLACEEVENTTYPE.2` | | Every place, as a comparison class | `EVERYPLACE` | `ABOVE(PLACESCALARDIM.1, MEAN(PLACECATEGORY.1), EVERYPLACE, 1.0)` | +| Pair fluent, and its literals (stage c2a) | `PAIRFL.`, `PAIRFL.(x, y)`, `SOME.PAIRFL.(x)`, `SOMEOF.PAIRFL.(x)`, `SAME.PAIRFL.(x, y)` | `PAIRFL.2(agent, patient)`, `NOT SOME.PAIRFL.3(agent)` | | Role prefix in a rule | `agent.`, `patient.` | `patient.PROPERTY.7` | | Episode (a scene), event, time point | `SCENE.`, `SCENE..EVENTINSTANCE.`, `TIME.` | `SCENE.8.EVENTINSTANCE.5` | ### Where the world comes from -`define` runs the taxonomy generator first, in memory, and writes the run to `taxonomy/` inside the world's folder, byte-identical to the taxonomy run alone with the same configuration and seed. The taxonomy supplies the entities' static facts and the feature rules. The world package then draws, from streams of its own, the one-place requirements (`world:requirements`), the event-type tree and its constraints (`world:event_tree`, `world:constraints`, `world:pairs`), the fluents and their rules (`world:fluents`), the initial values (`world:initial`), the preconditions (`world:preconditions`), the effects (`world:effects`), the conditions (`world:conditions`), and, since stage b2, the numeric fluents' drift (`world:numeric`), their initial values (`world:numeric_initial`), and the numeric effects, precondition literals, and condition literals (`world:numeric_effects`, `world:numeric_preconditions`, `world:numeric_conditions`), and runs the statistics episodes (`world:stats`). Since stage c1 the place taxonomy runs with the seed of `world:places` (so it never repeats the entities' draws) and is written to `places/`, and the place event types' requirements, and the samples of their capacities' local test, come from `world:place_event_types`; both streams were created after every earlier one. Separate streams mean that changing the fluent, precondition, or effect settings never changes the taxonomy, the requirements, the capacities, or the relation statistics; that with `numeric_fluents.count: 0` a world is the world of stage b1, byte for byte, and with the numeric shares at 0 its Boolean dynamics are those of the world without numeric fluents; and that with `places: null` a world is the world before places, byte for byte. `TAXONOMY.md`, "Where CAN features and verbs went", describes the requirements and the relation statistics in detail. +`define` runs the taxonomy generator first, in memory, and writes the run to `taxonomy/` inside the world's folder, byte-identical to the taxonomy run alone with the same configuration and seed. The taxonomy supplies the entities' static facts and the feature rules. The world package then draws, from streams of its own, the one-place requirements (`world:requirements`), the event-type tree and its constraints (`world:event_tree`, `world:constraints`, `world:pairs`), the fluents and their rules (`world:fluents`), the initial values (`world:initial`), the preconditions (`world:preconditions`), the effects (`world:effects`), the conditions (`world:conditions`), and, since stage b2, the numeric fluents' drift (`world:numeric`), their initial values (`world:numeric_initial`), and the numeric effects, precondition literals, and condition literals (`world:numeric_effects`, `world:numeric_preconditions`, `world:numeric_conditions`), and runs the statistics episodes (`world:stats`). Since stage c1 the place taxonomy runs with the seed of `world:places` (so it never repeats the entities' draws) and is written to `places/`, and the place event types' requirements, and the samples of their capacities' local test, come from `world:place_event_types`; both streams were created after every earlier one. Since stage c2a the initial pairs of the pair fluents come from `world:pair_initial`, after a stream `world:pair_fluents` from which stage c2b will draw the profiles and rates (nothing is drawn from it yet); both come after the place streams, so a world without pair fluents is the world of stage c1, byte for byte (its `config.yaml` records the two new seeds). Separate streams mean that changing the fluent, precondition, or effect settings never changes the taxonomy, the requirements, the capacities, or the relation statistics; that with `numeric_fluents.count: 0` a world is the world of stage b1, byte for byte, and with the numeric shares at 0 its Boolean dynamics are those of the world without numeric fluents; and that with `places: null` a world is the world before places, byte for byte. `TAXONOMY.md`, "Where CAN features and verbs went", describes the requirements and the relation statistics in detail. ### Rules as matrices, and the rule-set identity @@ -99,7 +101,7 @@ Every command runs from the root of the repository. With the repository's virtua PYTHONPATH=python python -m semantic_world.world define data/world/tiny.yaml [--seed N] [--out DIR] ``` -Generates a world and writes its folder (see "Reading the outputs"). The run embeds the taxonomy run and, since stage c1, the place taxonomy run, draws the fluents (Boolean and numeric), the event types, and the place event types, builds the definition, runs the agreement test (over every entity, every ordered pair of entities, every place, and every entity paired with every place) and the checks of the dynamics (no requirement reads a fluent, no effect writes a derived fluent, no contradiction, every precondition literal achievable, every numeric threshold inside its range), and runs 1,000 statistics episodes. An event type that is never legal in the statistics episodes has its own precondition literals drawn again, up to a few times; the command prints a note for each redraw, and a warning for each event type that is still never legal. The default world takes about 6 seconds (6.4 s on October 16, 2026, stage c1; 6.3 s in stage b3b, 6.1 s in stage b3a, 6.7 s in stage b2, 7.6 s in stage b1), of which the static side, the place taxonomy and the place event types included, is about a second. Every stage records this time, measured with nothing else running, and flags a growth of more than half in its proposal file; there is no hard time limit, and no step is made cheaper by changing what it produces by default (Jon's ruling 3 on stage a8). +Generates a world and writes its folder (see "Reading the outputs"). The run embeds the taxonomy run and, since stage c1, the place taxonomy run, draws the fluents (Boolean and numeric), the event types, and the place event types, reads the pair fluents of the event file and draws their initial pairs (stage c2a), builds the definition, runs the agreement test (over every entity, every ordered pair of entities, every place, and every entity paired with every place) and the checks of the dynamics (no requirement reads a fluent, no effect writes a derived fluent, no contradiction, every precondition literal achievable, every numeric threshold inside its range, and the rules of pair literals and effects under "The event file"), and runs 1,000 statistics episodes. An event type that is never legal in the statistics episodes has its own precondition literals drawn again, up to a few times; the command prints a note for each redraw, and a warning for each event type that is still never legal. The default world takes about 7 seconds (6.8 s on October 21, 2026, stage c2a; 6.4 s in stage c1, 6.3 s in stage b3b, 6.1 s in stage b3a, 6.7 s in stage b2, 7.6 s in stage b1), of which the static side, the place taxonomy and the place event types included, is about a second. Every stage records this time, measured with nothing else running, and flags a growth of more than half in its proposal file; there is no hard time limit, and no step is made cheaper by changing what it produces by default (Jon's ruling 3 on stage a8). ### `simulate` @@ -141,7 +143,7 @@ With `--places` (stage c1) the table is of the places, by default `RUN/views/pla PYTHONPATH=python python -m semantic_world.world check-fixtures [tests/fixtures/world] ``` -Runs the Python runtime and then the brute-force evaluator on every conformance fixture of the folder, prints one line per fixture, and exits with status 1 at the first failure. On the committed fixtures it ends with `33 fixtures passed the runtime and the brute-force evaluator`. See "The conformance fixtures" below. +Runs the Python runtime and then the brute-force evaluator on every conformance fixture of the folder, prints one line per fixture, and exits with status 1 at the first failure. On the committed fixtures it ends with `43 fixtures passed the runtime and the brute-force evaluator`. See "The conformance fixtures" below. ``` PYTHONPATH=python python -m semantic_world.world make-fixtures data/world/tiny.yaml [--out tests/fixtures/world] [--count N] [--steps N] @@ -169,6 +171,7 @@ A run writes one folder: | `definition.json` | The definition: every symbol, every literal, every rule in Boolean form, the matrix form, and every event type, the place event types included. | | `entities.csv` | One row per entity: `label`, `leaf`, the free PROPERTY and PART features, the scalars (full precision), the initial value of every base Boolean fluent, and the initial value of every numeric fluent (4 decimals since stage c1). | | `places.csv` | One row per place (stage c1): `label`, `leaf`, the free place features, and the place scalars (full precision); absent with `places: null`. | +| `initial_pairs.csv` | One row per ordered pair that holds at `TIME.1` (stage c2a): `pair_fluent`, `first`, `second` (both directions of a symmetric pair), by pair fluent, then by first and second in entity order; absent in a world without pair fluents. | | `derived/` | The derived values, each file tagged in `manifest.yaml` with the rule-set identity that produced it. | | `world_stats.yaml` | The world's statistics (below). | | `views/` | The views written by `view`, each with its sidecar. | @@ -180,13 +183,14 @@ The language-neutral part of the definition, with no code in it. Its top-level k - `version`: the format's version, 1; - `rule_set_id`: the rule-set identity; -- `symbols`: one entry per feature, scalar, fluent, numeric fluent, event type, and event-type category: `label`, `kind` (`property`, `part`, `scalar`, `fluent`, `numeric_fluent`, `event_type`, `event_type_category`, and, with places, `place_property`, `place_part`, `place_scalar`, `place_event_type`), `derived`, `fluent`, `arity`, for a base fluent its `initial_rate`, and for a numeric fluent its `range`, its `initial` distribution, and its `drift`; -- `literals`: everything the rules read, each with its `index`, its `key`, its `kind` (`feature`, `threshold`, `fluent`, `numeric_threshold`, `comparison`, `constraint`), its `role` (null for the entity itself, `agent`, `patient`, or `binding`), and the fields of its kind; +- `symbols`: one entry per feature, scalar, fluent, numeric fluent, pair fluent, event type, and event-type category: `label`, `kind` (`property`, `part`, `scalar`, `fluent`, `numeric_fluent`, `event_type`, `event_type_category`, with places `place_property`, `place_part`, `place_scalar`, `place_event_type`, and with pair fluents `pair_fluent`), `derived`, `fluent`, `arity`, for a base fluent its `initial_rate`, for a numeric fluent its `range`, its `initial` distribution, and its `drift`, and for a pair fluent (stage c2a, arity 2) its profile, `symmetry`, `functional`, and `total`, and its `initial_rate`; +- `literals`: everything the rules read, each with its `index`, its `key`, its `kind` (`feature`, `threshold`, `fluent`, `numeric_threshold`, `comparison`, `constraint`, and, since stage c2a, `pair_some` and `pair_someof`, the `SOME` and `SOMEOF` literals of a derived fluent's rule about its own entity, computed from the state's pairs before the matrices), its `role` (null for the entity itself, `agent`, `patient`, or `binding`), and the fields of its kind; - `rules`: every rule's `output`, `scope` (`entity` or `binding`; with places also `place`, a determined place feature, and `place_binding`, a place event type's constraint or requirement over an entity and a place), `family`, `inputs` (literal indices), `truth_table`, and `expression`. A derived feature and a derived fluent are entity rules; a constraint and a requirement are binding rules; - `layers`: the matrix form, written sparsely: for each layer its `terms` (the literals of one term of the rule's minimal DNF, whether each is complemented, and the threshold) and its `outputs`; -- `event_types`: one entry per event type and category, and per place event type (kind `place_event_type`, arity 2, its patient a place, with an empty precondition and no effects in stage c1): `label`, `kind`, `arity`, `roles`, `parent`, `level`, `features`, `explicit`, the `requirement` (its constraints, the rule that is the requirement, and the expression), the `precondition` (its literals, each a `role`, `fluent`, and `value`, or, for a numeric threshold literal, a `role`, `fluent`, `operator` (`>` or `<=`), and `threshold`; and the expression), and the `effects` (each a `role`, `fluent`, `value`, and expression, or, for a numeric effect, a `role`, `fluent`, `operator` (`+=` or `-=`), `amount`, and expression; and, for a conditional effect, a `condition` with its literals, each a `role`, `kind` (`feature`, `fluent`, or `numeric`), `symbol`, and `value` or `operator` and `threshold`, and its expression; an unconditional effect has no `condition` key). +- `event_types`: one entry per event type and category, and per place event type (kind `place_event_type`, arity 2, its patient a place, with an empty precondition and no effects in stage c1): `label`, `kind`, `arity`, `roles`, `parent`, `level`, `features`, `explicit`, the `requirement` (its constraints, the rule that is the requirement, and the expression), the `precondition` (its literals, each a `role`, `fluent`, and `value`, or, for a numeric threshold literal, a `role`, `fluent`, `operator` (`>` or `<=`), and `threshold`, or, since stage c2a, a pair literal with a `kind` (`pair`, `pair_some`, `pair_someof`, `pair_same`), its `pair_fluent`, its roles (`first` and `second`, or `role`), its `value`, and, for a compiled literal, its `origin` (`profile`); and the expression), and the `effects` (each a `role`, `fluent`, `value`, and expression, or, for a numeric effect, a `role`, `fluent`, `operator` (`+=` or `-=`), `amount`, and expression, or, for a pair effect, a `kind` (`pair`), `pair_fluent`, `first`, `second`, `value`, and expression; and, for a conditional effect, a `condition` with its literals, each a `role`, `kind` (`feature`, `fluent`, or `numeric`), `symbol`, and `value` or `operator` and `threshold`, or a pair literal as in a precondition, and its expression; an unconditional effect has no `condition` key); +- `spatial` and `spatial_id` (stage c2a), only when the event file gave a `spatial` block: for a pair fluent, a `detector` name and its `parameters`, and the SHA-256 hash of the block's canonical JSON. Neither is part of the rule-set identity; the Python runtime checks the block's form and nothing else. -Floating-point numbers are written with 17 significant digits, so both runtimes read the same values. With `places: null`, no place symbol, literal, rule, layer, or event type is written, so the definition and its identity are those of the world before places. `tests/fixtures/world/README.md` describes every field, and how the layers are evaluated. +Floating-point numbers are written with 17 significant digits, so both runtimes read the same values. With `places: null`, no place symbol, literal, rule, layer, or event type is written, so the definition and its identity are those of the world before places; without pair fluents, no pair symbol, literal, record, or `spatial` key is written, so the definition is that of stage c1. `tests/fixtures/world/README.md` describes every field, and how the layers are evaluated. ### `derived/` @@ -213,7 +217,7 @@ Nothing in `derived/` is an input to anything, and nothing there is edited by ha - `fluents`: the counts of base and derived Boolean fluents. - `event_types`: the counts of one-place event types, two-place event types, and categories. -- `precondition_literals` and `effects`: the totals, by kind of event type, by role, and how many come from event-type features (numeric literals and effects counted among them); under `effects`, the `conditional` effects by kind of event type and the `condition_literals` by symbol kind (`feature`, `fluent`, or `numeric`), by role, and from event-type features (stage b1). +- `precondition_literals` and `effects`: the totals, by kind of event type, by role, and how many come from event-type features (numeric literals and effects counted among them); under `effects`, the `conditional` effects by kind of event type and the `condition_literals` by symbol kind (`feature`, `fluent`, or `numeric`, and `pair` when the world has a pair condition literal, stage c2a), by role, and from event-type features (stage b1). - `fix_ups`: the effects, literals, and conditions redrawn or dropped while the dynamics were drawn (a contradiction, a duplicate, an effect that changes nothing, a literal that nothing can make true, a condition that breaks a rule of "Conditional effects": `conditions_redrawn` and `conditions_dropped`; `numeric_literals_redrawn`, the numeric literals drawn again because they were not achievable or their fluent was taken). - `enabling_graph`: the edges from an event type whose effect produces a value to an event type whose precondition requires it (a numeric effect that adds enables a literal `>`, one that subtracts a literal `<=`), and the longest chain. - `absorbing_fluents`: the base Boolean fluents that some effect sets to one value and no effect sets back. @@ -238,7 +242,7 @@ A history is one JSON object per episode: its `label`, its `seed` entity, its `p "final": "TIME.5", "quiescent": false} ``` -`changes` lists only the base fluents whose value changed: an effect that sets a fluent to the value it already has records nothing, an effect whose condition failed records nothing, and a derived fluent's change is never recorded, because it is recomputed. In a world with numeric fluents (stage b2), a history also holds `initial_numeric` (each participant's numeric values at `TIME.1`), an event's `changes` hold the numeric changes its effects made, each `{"entity", "fluent", "from", "to", "cause"}` with the event's own label as its cause, judged after the effect and before the drift, and a step holds the drift's changes to the participants under `drift`, each with `"cause": "drift"`; so every numeric change is recorded with its cause, and a reader can rebuild the values at every time point from the history alone. A world without numeric fluents writes none of these keys. The corpus's `scenes.jsonl` holds the same schema, and the 3D engine will write it too. `tests/fixtures/world/README.md`, "Histories", gives every field, and "The order within a step" the arithmetic a runtime must reproduce bit for bit. +`changes` lists only the base fluents whose value changed: an effect that sets a fluent to the value it already has records nothing, an effect whose condition failed records nothing, and a derived fluent's change is never recorded, because it is recomputed. In a world with numeric fluents (stage b2), a history also holds `initial_numeric` (each participant's numeric values at `TIME.1`), an event's `changes` hold the numeric changes its effects made, each `{"entity", "fluent", "from", "to", "cause"}` with the event's own label as its cause, judged after the effect and before the drift, and a step holds the drift's changes to the participants under `drift`, each with `"cause": "drift"`; so every numeric change is recorded with its cause, and a reader can rebuild the values at every time point from the history alone. A world without numeric fluents writes none of these keys. In a world with pair fluents (stage c2a), a history also holds `initial_pairs` after `initial_numeric`, every ordered pair that holds among its participants at `TIME.1` as `[pair fluent, first, second]` triples (an episode keeps the pairs among its participants and no other, WM.38), and an event's `changes` hold its pair changes, each `{"pair": "PAIRFL.1", "first": ..., "second": ..., "to": true}`, a symmetric pair once in the direction the effect names; `replay` writes both directions and reproduces every pair at every time point. A world without pair fluents writes neither key. The corpus's `scenes.jsonl` holds the same schema, and the 3D engine will write it too. `tests/fixtures/world/README.md`, "Histories", gives every field, and "The order within a step" the arithmetic a runtime must reproduce bit for bit. ## The configuration file @@ -317,9 +321,30 @@ event_types: An entry that breaks a rule of the world (an effect on a derived fluent, a requirement that reads a fluent, a contradiction, an unachievable literal, a numeric threshold outside its fluent's range, a condition that no able binding satisfies or that reads the effect's own Boolean fluent or a fluent the precondition fixes) is an error that names the file and the entry. `data/world/tiny_chain.yaml` uses the file with the tiny world. +**Pair fluents in the event file** (stage c2a). A `pair_fluents` block defines pair fluents between entities, `PAIRFL.1` and on without a gap, each with its profile and its initial rate; the preconditions and conditions may then hold pair literals, the effects may add or delete pairs, and a `spatial` block may give a detector for a pair fluent. `data/world/events/pair_example.yaml` defines four pair fluents (one of each symmetry, two of them functional) and five event types over them, and `data/world/tiny_pairs.yaml` uses it with the tiny world: + +```yaml +pair_fluents: + PAIRFL.1: {symmetry: symmetric, functional: false, initial_rate: 0.2} # say, near + PAIRFL.3: {symmetry: general, functional: true, initial_rate: 0.2} # say, follows +event_types: + EVENTTYPE2.1.1: # approach + precondition: "NOT PAIRFL.1(agent, patient)" + effects: ["PAIRFL.1(agent, patient) := 1"] + EVENTTYPE2.1.2: # follow + precondition: "PAIRFL.1(agent, patient)" + effects: + - "PAIRFL.3(agent, patient) := 1" + - "when SOME.PAIRFL.2(patient) then patient.BOOLFL.1 := 0" +spatial: + PAIRFL.1: {detector: within, parameters: {distance: 1.0}} +``` + +The compiled literals are added to explicit entries too: `EVENTTYPE2.1.2` gains `NOT SOME.PAIRFL.3(agent)`, because `PAIRFL.3` is functional. An entry that breaks a rule of "Pair fluents" is an error that names the file and the entry: a `SAME` literal on a pair fluent that is not functional (or on a symmetric functional one, which no two entities can share), a pair literal that is not achievable (a positive literal needs a positive initial rate or some add; a negative one an initial rate below 1, a functional pair fluent, or some delete), two effects on the same pair (both directions of a symmetric pair are one pair), an add whose precondition already requires the pair or a delete whose precondition already requires its absence, a conditional add on an asymmetric or functional pair fluent, a pair condition that names a pair or a row the precondition (its compiled literals included) names or the pair the effect itself writes, and compiled literals that contradict the entry's own precondition. A one-place event type reads `SOME` and `SOMEOF` of its agent only. A location fluent (`total: true`) and a `place_event_types` entry come in stage c3 and are errors here. + ## The default world -The figures of `data/world/default.yaml` with seed 1, from a run of October 16, 2026 (stage c1); `define` took 6 seconds (6.4 s; 6.3 s in stage b3b, 6.1 s in stage b3a, 6.7 s in stage b2, 7.6 s in stage b1, 7.8 s in stage a8). +The figures of `data/world/default.yaml` with seed 1, from a run of October 21, 2026 (stage c2a, whose definition and identity are those of stage c1); `define` took 7 seconds (6.8 s; 6.4 s in stage c1, 6.3 s in stage b3b, 6.1 s in stage b3a, 6.7 s in stage b2, 7.6 s in stage b1, 7.8 s in stage a8). | What | Figure | | --- | --- | @@ -405,7 +430,7 @@ What each lever does, against the baseline's share of 0.41 over seeds 1 to 3: The Rust runtime, when it is built, must give the same results as the Python runtime. The conformance fixtures in `tests/fixtures/world/` are its tests, written now. Each fixture is one JSON file with a complete definition inline, the entities' static facts, the initial state, and a list of steps; each step gives its events and what must come out: the legal bindings of every event type before the step, and the state and the derived fluents after it. A fixture can end in an error that `apply` must raise, an illegal event or interfering events. -Two kinds are committed: twenty-seven hand-written fixtures over a hand world of three entities, with every expected value worked out by hand (a fluent with no event; an effect that sets, clears, or leaves a fluent; a derived fluent that changes; a precondition that blocks; two events that do not interfere; the kinds of interference; since stage b1, a static condition that holds and one that fails, a condition on a derived fluent that holds and one that fails, and interference through a condition; and, since stage b2, the comparisons, with ties, `ENTITY`, and a category mean, a numeric effect, the drift, clamping at both ends of the range, an effect and the drift in one step, a numeric precondition at its threshold, a numeric condition, and interference through a numeric fluent), and six generated from the tiny world, numeric fluents included, with expected values from a brute-force evaluator that reads the truth tables and never the matrices. `check-fixtures` runs every fixture through the runtime and then through the evaluator, so the two implementations check each other. `tests/fixtures/world/README.md` is written for whoever builds the Rust runtime: the fixture format, the definition record, the canonical JSON behind the rule-set identity, how literals and layers are evaluated, the semantics of each operation, the order within a step and the float format, and the comparisons. +Two kinds are committed: thirty-seven hand-written fixtures over a hand world of three entities, with every expected value worked out by hand (a fluent with no event; an effect that sets, clears, or leaves a fluent; a derived fluent that changes; a precondition that blocks; two events that do not interfere; the kinds of interference; since stage b1, a static condition that holds and one that fails, a condition on a derived fluent that holds and one that fails, and interference through a condition; since stage b2, the comparisons, with ties, `ENTITY`, and a category mean, a numeric effect, the drift, clamping at both ends of the range, an effect and the drift in one step, a numeric precondition at its threshold, a numeric condition, and interference through a numeric fluent; and, since stage c2a, over three pair fluents, a symmetric add and a symmetric delete that write both directions, an asymmetric add and a functional add blocked by their compiled literals, a derived fluent over `SOME` that changes with a pair, interference through the same pair, through a functional row, and through both directions of an asymmetric pair, and a pair condition that holds and one that fails), and six generated from the tiny world, numeric fluents and places included, with expected values from a brute-force evaluator that reads the truth tables and never the matrices (the pair sets directly, never the matrices, since stage c2a). A fixture of a world with pair fluents carries `initial_pairs` beside `initial` and `pairs` after each step. `check-fixtures` runs every fixture through the runtime and then through the evaluator, so the two implementations check each other. `tests/fixtures/world/README.md` is written for whoever builds the Rust runtime: the fixture format, the definition record, the canonical JSON behind the rule-set identity, how literals and layers are evaluated, the semantics of each operation, the order within a step and the float format, and the comparisons. ## Using the package from Python @@ -418,7 +443,7 @@ result = define(config) result.write("runs/world/tiny_seed7") ``` -`result` holds everything the run produced: `result.taxonomy` (the embedded taxonomy run), `result.statics` (the static side: the one-place requirements, the event-type tree, the relations, the capacities, and the relation statistics), `result.fluents`, `result.numeric` (the numeric fluents, with their ranges, distributions, drifts, and initial values), `result.event_types` (every event type with its requirement, precondition, and effects), `result.places` (the place taxonomy run, the place event types with their requirements over every binding, and their capacities; None with `places: null`), `result.definition` (the definition, with `result.rule_set_id`), `result.stats` (the contents of `world_stats.yaml`), and `result.warnings`. `config_from_mapping` in the same module builds a configuration from a dictionary, for sweeping a setting in code. +`result` holds everything the run produced: `result.taxonomy` (the embedded taxonomy run), `result.statics` (the static side: the one-place requirements, the event-type tree, the relations, the capacities, and the relation statistics), `result.fluents`, `result.numeric` (the numeric fluents, with their ranges, distributions, drifts, and initial values), `result.event_types` (every event type with its requirement, precondition, and effects), `result.places` (the place taxonomy run, the place event types with their requirements over every binding, and their capacities; None with `places: null`), `result.pair_fluents` (the pair fluents and their initial pairs, stage c2a; empty without an event file that defines them), `result.definition` (the definition, with `result.rule_set_id`), `result.stats` (the contents of `world_stats.yaml`), and `result.warnings`. `config_from_mapping` in the same module builds a configuration from a dictionary, for sweeping a setting in code. The runtime works on a loaded definition: @@ -431,7 +456,7 @@ state = initial_state(definition) legal = legal_bindings(definition, state, "EVENTTYPE2.1.2") # every legal binding, as pairs of entity indices ``` -`derive(definition, state, entities)` gives the derived facts, `able(definition, event_type, bindings)` and `legal(definition, state, event_type, bindings)` judge bindings (for a place event type, bindings of an entity index and a place index, since stage c1), `legal_bindings` lists every legal binding among given entities in a fixed order (and among given places, `legal_bindings(definition, state, event_type, entities, places)`, for a place event type), and `apply(definition, state, events)` gives the state after one step, or raises an `IllegalEventError` or an `InterferenceError` that names the event and the reason (an event of a place event type is refused in stage c1); `apply_step` returns the numeric values after the effects and before the drift beside the state. A `State` holds `values` (the Boolean fluents) and `numeric` (the numeric fluents, float64). The comparisons are `greater(definition, dimension, x, y, state)`, `above(definition, dimension, x, standard, z, state)` and `below`, and `increase(definition, fluent, x, before, after)` and `decrease`, over entities by index or `Mean(category)`, with `ENTITY` and `RANGE` as the symbols; `comparison_classes(definition)` gives every class's count, mean, and standard deviation. The runtime draws no random number and chooses no event. Episodes are run by `semantic_world.world.episodes.simulate(folder, episodes, seed, settings)`, or by an `EpisodeGenerator` over a definition in memory; `semantic_world.world.history.replay(definition, history)` checks a history against the runtime, every recorded numeric value to the bit, and returns its states. +`derive(definition, state, entities)` gives the derived facts, `able(definition, event_type, bindings)` and `legal(definition, state, event_type, bindings)` judge bindings (for a place event type, bindings of an entity index and a place index, since stage c1), `legal_bindings` lists every legal binding among given entities in a fixed order (and among given places, `legal_bindings(definition, state, event_type, entities, places)`, for a place event type), and `apply(definition, state, events)` gives the state after one step, or raises an `IllegalEventError` or an `InterferenceError` that names the event and the reason (an event of a place event type is refused in stage c1); `apply_step` returns the numeric values after the effects and before the drift beside the state; a `ProfileViolationError` after a step marks a broken definition (stage c2a). A `State` holds `values` (the Boolean fluents), `numeric` (the numeric fluents, float64), and `pairs` (one Boolean matrix per pair fluent, stage c2a; `State.from_true` takes the pairs as label triples, and `pairs_record` gives them back). The comparisons are `greater(definition, dimension, x, y, state)`, `above(definition, dimension, x, standard, z, state)` and `below`, and `increase(definition, fluent, x, before, after)` and `decrease`, over entities by index or `Mean(category)`, with `ENTITY` and `RANGE` as the symbols; `comparison_classes(definition)` gives every class's count, mean, and standard deviation. The runtime draws no random number and chooses no event. Episodes are run by `semantic_world.world.episodes.simulate(folder, episodes, seed, settings)`, or by an `EpisodeGenerator` over a definition in memory; `semantic_world.world.history.replay(definition, history)` checks a history against the runtime, every recorded numeric value to the bit, and returns its states. ## Recipes @@ -442,6 +467,7 @@ legal = legal_bindings(definition, state, "EVENTTYPE2.1.2") # every legal bind - **A world without change.** `fluents: {count: 0}` and `numeric_fluents: {count: 0}`. Every able binding is legal at every time point, histories record no change, and `scene.policy: uniform_event_type` in the corpus then gives the scenes of the corpus's old scene generator. - **A world without places.** `places: null` (and no `place_event_types` block): the world before stage c1, byte for byte, with no `places/`, `places.csv`, or `place_capacities.csv`. A study that wants other kinds of place names its own taxonomy configuration with `labels.kind: place` under `places.taxonomy`; `place_event_types.constraint_families: {comparison: 1}` makes every place requirement a comparison of an entity scalar with a place scalar. - **A known chain of events.** Write an event file (above), and name it under `event_types.event_file`. `simulate` on the world shows the chain's events in the order the preconditions force. +- **A world with pair fluents.** `data/world/tiny_pairs.yaml`, or an event file with a `pair_fluents` block (above). `simulate` records every pair change, and `initial_pairs.csv` the pairs at `TIME.1`. The corpus refuses such a world until stage c4. - **More two-place events.** Lower `event_types.binary.features.expected_true`, or weigh the two-place event types in `scene.event_type_weights` of the corpus configuration; see "The two-place share" for every lever. - **A model's input.** `view --preset classic` for the columns of the old `instances.csv`; `view --preset base --include ISA` for the base vector with the categories. Join `derived/capacities.csv` on `label` for the two-place capacities. - **Legality apart from occurrence.** `simulate --legal` records, at every step, how many bindings of each event type were legal, beside the events that happened. @@ -452,7 +478,8 @@ legal = legal_bindings(definition, state, "EVENTTYPE2.1.2") # every legal bind - **"never able".** An event type whose requirement holds for no entity or no pair gets no events and no word. The world keeps it; another seed, or looser `event_types.unary.rules`, gives another requirement. - **A derived file is refused.** The file's rule-set identity differs from the definition's. The run folder holds files of two runs; regenerate it. - **The agreement test fails.** The error names the rule and the first disagreement. The two forms of one rule set must agree, so this is a bug in the generator, not in the configuration. Keep the configuration and seed and report it. -- **An event file entry is refused.** The error names the file and the entry: an effect on a derived fluent, a requirement that reads a fluent, a contradiction, a literal that nothing can make true, a numeric threshold outside its range, or a condition that breaks a rule (no able binding satisfies it, or it reads the effect's own Boolean fluent or a fluent the precondition fixes). +- **An event file entry is refused.** The error names the file and the entry: an effect on a derived fluent, a requirement that reads a fluent, a contradiction, a literal that nothing can make true, a numeric threshold outside its range, a condition that breaks a rule (no able binding satisfies it, or it reads the effect's own Boolean fluent or a fluent the precondition fixes), or, since stage c2a, a pair literal or effect that breaks a rule of "Pair fluents in the event file". +- **"pair fluents enter the corpus in stage c4".** The corpus refuses a world with pair fluents, so that no document states a fact its oracle cannot check. Use a world whose event file defines none. - **Few events in a world of needs.** A numeric literal `> t` on a fluent that nobody starts above and that drifts downward, with a small effect, is achievable but rarely true, so its event type is rarely legal. `world_stats.yaml` (`episodes.legal_share`) shows it; a positive drift, a `fixed` or `beta` initial distribution, or a larger `numeric_amounts` interval helps. - **`taxonomy.config` is not found.** Paths in a configuration are read from the folder the command runs in. Run from the root of the repository, as the examples do. @@ -460,7 +487,7 @@ legal = legal_bindings(definition, state, "EVENTTYPE2.1.2") # every legal bind - `docs/specs/WORLD_AND_LANGUAGE.md`: the design, with "Terms", "Architecture", "The world definition", "Phase (a): the world", "Phase (a): determinism", and the build stages. - `tests/fixtures/world/README.md`: the fixture format, the definition record, the canonical JSON, and the semantics of the runtime, for the Rust runtime. -- `docs/proposals/2026-10-07-world-stage-a1-decisions.md` to `2026-10-16-world-stage-c1-decisions.md`: the engineering choices of each stage, and Jon's rulings on them. -- `docs/specs/WORLD_PHASE_C.md`: phase (c), with the as-built notes of stage c1. -- `docs/DECISIONS.md`, "World model": WM.1 to WM.60 and WM.E1 and following. +- `docs/proposals/2026-10-07-world-stage-a1-decisions.md` to `2026-10-21-world-stage-c2a-decisions.md`: the engineering choices of each stage, and Jon's rulings on them. +- `docs/specs/WORLD_PHASE_C.md`: phase (c), with the as-built notes of stages c1 and c2a. +- `docs/DECISIONS.md`, "World model": WM.1 to WM.62 and WM.E1 and following. - `docs/guides/TAXONOMY.md`: the taxonomy inside the world, and "Where CAN features and verbs went". `docs/guides/CORPUS.md`: the documents written about a world. diff --git a/docs/proposals/2026-10-21-world-stage-c2a-decisions.md b/docs/proposals/2026-10-21-world-stage-c2a-decisions.md new file mode 100644 index 0000000..43d4a41 --- /dev/null +++ b/docs/proposals/2026-10-21-world-stage-c2a-decisions.md @@ -0,0 +1,163 @@ +# Proposal: decisions for stage c2a of the world-and-language refactor + +October 21, 2026. Raised in the orientation for stage c2a of `docs/specs/WORLD_PHASE_C.md` ("c2a. Pair fluents: the definition, the runtime, and the fixtures", the second stage of phase (c)), and while building the stage. Status: working design. Every choice below is Claude Code's unless Jon changes it. The choices are logged as WM.E256 and following in `docs/DECISIONS.md`; Jon's rulings on stage c1's open questions are WM.61 and WM.62. + +Stage c2a builds, in this order: the rounding of the taxonomy's similarity range (Jon's ruling 1 on stage c1, WM.61); pair fluents between entities as an event file gives them (their symbols and profiles in `definition.json`, the pair literals in the literal table, the streams `world:pair_fluents` and `world:pair_initial`, and `initial_pairs.csv`); the runtime's state, `derive`, `legal`, and `apply` over pairs, with the compiled literals, the symmetric writes, the profile invariant, and non-interference over pairs, rows, and columns; the `spatial` block and `spatial_id`; episodes, histories, and `replay` with pair changes; the brute-force evaluator and the fixtures over pairs, with hand-written fixtures 1 to 4, 10, 12 to 14, and 16; a world with pair fluents for the tests; and the corpus's refusal of a world with pair fluents. Nothing is generated yet: with no event file that defines pair fluents, a world has none, and is byte-identical to stage c1. + +## Jon's rulings on stage c1's open questions + +Jon's rulings of October 21, 2026, on the five open questions of `docs/proposals/2026-10-16-world-stage-c1-decisions.md`: + +1. **The taxonomy's similarity range is rounded (WM.61).** On Linux (x86_64) with the same Python and package versions as Jon's Mac, every output of every world and corpus now matches the Mac's, and every taxonomy output matches except `summary.yaml` in the configurations whose superordinates have a similarity bound (`default`, `relations`, `rule_file`): `superordinates.similarity_min` and `similarity_max` differ in the 14th digit (for example -0.07165743639000198 on the Mac against -0.07165743639000184 on Linux). They come from a matrix product, which the two machines' math libraries compute differently. The CSVs are written at 6 decimals and match. Ruling: round `similarity_min` and `similarity_max` to 6 decimals when `summary.yaml` is written, as the CSVs are. Built as item 1 of this stage. Extended on the same day, after the stage was committed: a rounded value of zero is written as `0.0`, never `-0.0`, because the tiny taxonomies' similarity is about -1.24e-17, arithmetic noise around zero, whose sign is exactly the last-digit noise the rounding was meant to remove (the follow-up below). +2. **Numeric threshold points stay unrounded.** They are computed by plain arithmetic, which gives the same result on every machine. No change. +3. **The place constraints keep borrowing the key-lock and comparison settings of `event_types.binary`** until a world needs them apart. No change. +4. **The tiny world's place requirements (WM.62).** In stage c3, `data/world/tiny.yaml`'s `place_event_types.constraint_families` is changed so that every tiny place requirement reads the place (for example `{patient: 1, cross: 1, key_lock: 1}`, with the agent family at 0), because c3's tests of habitats and of acts at a place need tiny place requirements that read the place. Not built in this stage: WM.62 is recorded in `docs/DECISIONS.md`, and a note on the c3 entry of "Build stages" in `WORLD_PHASE_C.md` tells the c3 session to build it. +5. **The level rule of the relation proportions** (WM.E251, c1's choice 12) is accepted as built. No change. + +## Orientation + +### What was read + +This is a fresh session. It read `CLAUDE.md` in full; `docs/specs/WORLD_PHASE_C.md` in full; `docs/specs/WORLD_AND_LANGUAGE.md` ("A principle for defaults", "The runtime's operations", "Conformance fixtures", "Labels", "Files", "Rules: Boolean form and matrices", "Rule-set identity and derived values", phase (a)'s "Event types", "Preconditions", "Effects", "Explicit event types", "Time, steps, and inertia", "Episodes and selection", "Histories", and "Phase (a): determinism", and phase (b)'s "Numeric fluents" and "Conditional effects"); `docs/proposals/2026-10-16-world-stage-c1-decisions.md` in full; `docs/DECISIONS.md` (the "World model" section WM.1 to WM.60, WM.E101 to WM.E111, WM.E193, WM.E240 to WM.E255, and "Process"); `docs/guides/WORLD.md` and `tests/fixtures/world/README.md`; the taxonomy's `analysis.py` (`summary_stats`) and `io.py`; the world package (`definition.py`, `matrices.py`, `dynamics.py`, `runtime.py`, `event_file.py`, `event_types.py`, `fluents.py`, `derived.py`, `generate.py`, `config.py`, `streams.py`, `episodes.py`, `history.py`, `fixtures.py`, `identity.py`, `places.py`, `policies.py`, `stats.py`, `numeric.py`, `__main__.py`); `python/semantic_world/figures.py` (the off settings and the baselines); the corpus's `world.py` where it reads a world; `data/world/events/chain_example.yaml` and the world configurations; `tests/world/` (`hand_world.py`, `test_world_runtime.py`, `test_world_fixtures.py`, `test_world_episodes.py`, `test_world_conditions.py`, `test_world_places.py`, `test_world_off_settings.py`, `test_world_identity.py`) and `tests/taxonomy/test_taxonomy_golden.py` with `make_golden_hashes.py`; and the hand-written fixtures. + +### How the pieces stand today + +Main holds stage c1 (a8e5046, merged as PR #32). The world draws from eighteen streams, `world:requirements` to `world:place_event_types`; the last two are c1's. The definition has four rule scopes (`entity`, `binding`, `place`, `place_binding`) over one literal table of six literal kinds, and symbol kinds for features, scalars, Boolean and numeric fluents, event types, and the place objects. A precondition is a tuple of `Literal` (a Boolean fluent of a role, or a numeric threshold), an effect an `Effect` (a Boolean or numeric fluent of a role, with an optional condition of `ConditionLiteral`s over static features, Boolean fluents, and numeric thresholds); precondition and condition literals live in the event-type records, not in the literal table, which holds the rules' inputs only. The runtime's `State` holds the Boolean values and the numeric values of every entity; `apply` collects each event's writes as `(entity, fluent)` keys and reads through the cones of derived fluents. Histories record Boolean and numeric changes; `replay` reproduces them. The brute-force evaluator reads records and truth tables, with a state of fluent sets and numeric values. The event file loader (`event_file.py`) reads `fluents` (Boolean and numeric) and `event_types` entries, parses a precondition through the taxonomy's expression parser, and checks against the generated world in `event_types.py` (`_Dynamics.apply_event_file`, `check_explicit_achievable`) and `check_dynamics`. + +The taxonomy's `summary_stats` (`analysis.py`) writes `superordinates.similarity_min` and `similarity_max` from `tree.superordinate_similarity`, a pair of floats from the similarity matrix of the superordinates' vectors, at full precision. + +Before the first change, a reference run of the tiny and default worlds and corpora was made from `main` at a8e5046 (`define` and `generate` of both configurations) into `/Users/jon/Documents/Projects/semantic-world-reference/c2a/`, outside the repository, with the commit and the log beside it. + +### Plan for the stage + +In the order of the scope: + +1. **The similarity range** rounded to 6 decimals where `summary.yaml` is written (`summary_stats`); the golden hashes rewritten; the a5a comparison against the rounded golden value; the check of the other YAML and JSON outputs for reals computed by a library function and written at full precision. +2. **Pair fluents in the definition**: a `pair_fluents.py` module with `PairFluent` (label, index, symmetry, functional, initial rate), the `pair_fluents` block of the event file, the symbol kind `pair_fluent` in `definition.json`, the pair literals (`pair`, `pair_some`, `pair_someof`, `pair_same`) as `PairLiteral` in `dynamics.py`, and the entity-scope pair literals in the literal table after the numeric thresholds. +3. **The streams** `world:pair_fluents` and `world:pair_initial` after `world:place_event_types`, the initial pairs by the specification's procedure, and `initial_pairs.csv`. +4. **The runtime**: `State.pairs`, `derive` over `SOME` and `SOMEOF`, `legal` over the four kinds, `apply` with the symmetric writes and the invariant, and non-interference over pairs, rows, and columns. +5. **The event file's preconditions, effects, and conditions** over pairs, with every rule of the specification as an error naming the file and the entry, and the compiled literals. +6. **The `spatial` block** and `spatial_id`. +7. **Episodes, histories, and `replay`** with initial pairs, redraws, and pair changes. +8. **The evaluator and the fixtures**: pairs in the brute-force evaluator, `initial_pairs` and `pairs` in the fixture format, the hand world extended with three pair fluents, and fixtures `hand_29` and following. +9. **The example world** `data/world/events/pair_example.yaml` with `data/world/tiny_pairs.yaml`, the corpus's refusal, the tests, the check list, and the documents. + +### Reading of the scope + +- **"The pair literals join the literal table."** The literal table holds the inputs of rules. In this stage a rule may read `SOME.PAIRFL.k` and `SOMEOF.PAIRFL.k` of its own entity, so the entity-scope literals of the kinds `pair_some` and `pair_someof` (role `null`) join the table when a rule reads them, computed before the matrices as threshold literals are. The pair literals of preconditions and conditions, of all four kinds and with their roles, live in the event-type records, as Boolean and numeric precondition and condition literals do, and are judged per binding outside the matrices; the loader accepts all four kinds in the literal table, so that a definition that lists them is well formed. (Choice 3 below.) +- **"Build the state so that c3 only adds the second kind."** `State.pairs` is a tuple with one Boolean matrix per pair fluent, in label order, whose rows are entities and whose columns are the fluent's targets: entities in this stage, and places for a location fluent in c3, where the matrix is `(entities, places)`. +- **"Nothing is generated yet."** With no `pair_fluents` block in the event file, nothing is drawn from the two new streams, and no symbol, literal, column, or key is written, so the definition, its identity, and every file but `config.yaml` (which lists the seeds of the two new streams) are those of stage c1. +- **"The corpus does not use pair fluents in this stage."** `load_world` refuses a world with pair fluents, so no corpus states a fact its oracle cannot check. + +### Questions + +No question of the orientation changes the design or a file format beyond what `WORLD_PHASE_C.md` and the stage prompt say. The stage proceeds. Open questions found while building are listed at the end. + +## Item 1: the similarity range rounded (WM.61) + +`summary_stats` in `python/semantic_world/taxonomy/analysis.py` rounds `superordinates.similarity_min` and `similarity_max` to 6 decimals when it builds the summary; a null range stays null, and the value the similarity bound is checked against (`tree.superordinate_similarity`) is not rounded, so no draw changes. Nothing else the taxonomy computes or writes changed. + +**Golden hashes rewritten** (`python tests/taxonomy/make_golden_hashes.py`): `summary.yaml` of `default`, `relations`, `rule_file`, `tiny`, and `tiny_relations` (and `tiny` and `tiny_relations` once more in the follow-up below). The two tiny configurations also have a similarity bound (their tiny values, -1.24e-17, round to -0.0). `places_default` and `places_tiny` did not move (their ranges, -0.1 and -0.5, were already short), and no other file of any configuration moved. `test_relabeled_outputs_equal_the_a5a_outputs_label_for_label` now compares the summary's `superordinates` with the golden folder's values rounded to 6 decimals; no golden folder was edited. + +**The check of the other outputs.** Every YAML and JSON file of the reference runs of the tiny and default worlds and corpora (`config.yaml`, `rules.yaml`, `summary.yaml`, `rule_matrices.json`, `derived/manifest.yaml` of both taxonomy runs; `definition.json`, `world_stats.yaml`, `config.yaml` of the worlds; `stats.yaml`, `config.yaml`, and the test-set and document files of the corpora) was scanned for a real written with more than 6 decimals. Only `summary.yaml` of the entity taxonomies had any: the similarity range, now rounded, and `mean_true_features_per_instance`, a mean of integer sums over the instances, which is exact in float64 (plain arithmetic) and stays as it is. The CSVs that hold the taxonomy's scalar draws at full precision (`entities.csv`, `places.csv`, WM.E33; c1's open question 1) are not YAML or JSON and were not in the scope of the check; they remain open below. + +**Figures.** No world or corpus figure moved: a world run copies the taxonomy's `summary.yaml` into `taxonomy/` and the place run's into `places/`, but no figure reads the range, and no rule-set identity changed. `python -m semantic_world.figures snapshot` rewrote the snapshot with 0 figures moved, and `figures guides --check` reported 0 figures that would move. + +**The follow-up: a rounded zero (Jon's ruling of October 21, 2026, extending WM.61).** The tiny taxonomies' range, about -1.24e-17, rounded to `-0.0`, whose sign depends on exactly the noise the rounding removes. `rounded_similarity` in `analysis.py` now rounds and adds `0.0` (`round(x, 6) + 0.0`), so a rounded zero is written as `0.0`; a test checks that -1e-17 (and the tiny value, and ±4e-7) comes out as a positive zero and that the tiny taxonomy's `summary.yaml` holds no `-0.0`. The golden hashes of `summary.yaml` moved again for `tiny` and `tiny_relations` alone (the five configurations of the first rounding keep their hashes for the other three). **Checked for the same fault, every other value the world or the taxonomy rounds when drawn or written (WM.60 and WM.61):** the drift and the numeric effect constants (`numeric.drawn`, `round(x, 4)`), the initial numeric values (`InitialDistribution.draw`, rounded then clamped to the range), the scalar thresholds of the taxonomy's rules and the world's requirements, constraints, and derived rules (`Threshold`, `model_quantile_threshold`, `Gt` and `Cmp` in `expressions.py`), and the comparison margins (`constraints.py`). Each of them could in principle round a value in (-0.00005, 0) to `-0.0` (a drift of -0.00004, a threshold at the median of a scalar centered at 0 drawn a hair below it, a margin quantile of 0.49999). Every YAML, JSON, and CSV output of every configuration in `data/` (the seven taxonomy configurations, the four world configurations with their embedded taxonomy and place runs, and the tiny and default corpora of `runs/`) was scanned for a `-0.0` token after the fix: none holds one, so, as the ruling says, the rule is applied to the similarity range alone. The statistics of `world_stats.yaml` and `stats.yaml` round shares and means when written (`round(x, 6)`); they were in the scan and hold none either. + +## Engineering choices made while building + +Numbered for `docs/DECISIONS.md` (WM.E256 to WM.E269; the last from the follow-up on the rounded zero). Each is Claude Code's choice inside the stage's scope. + +1. **The rounding** (above): 6 decimals when the summary is built; the golden hashes of five configurations rewritten; the a5a comparison against the rounded golden value; the check of the other outputs found nothing else. +2. **The streams.** `world:pair_fluents` and `world:pair_initial` follow `world:place_event_types` in `STREAM_NAMES`. Nothing is drawn from `world:pair_fluents` in this stage; the initial pairs come from `world:pair_initial`, fluent by fluent in label order. A world without an event file that defines pair fluents draws from neither, so its definition, identity, and every file but `config.yaml` (which lists the two new seeds under `provenance.stream_seeds`) are those of stage c1. +3. **`pair_fluents.py`.** `PairFluent(label, index, symmetry, functional, initial_rate, total=False)` and `PairFluents(fluents, initial_pairs)`, the initial pairs one Boolean matrix per fluent with the entities as rows and the fluent's targets as columns (entities now; places for a location fluent in c3, where the matrix is `(entities, places)`), so that c3 adds the second kind of target and nothing else. The event file's `pair_fluents` block must label its entries `PAIRFL.1` to `PAIRFL.k` without a gap (the specification numbers the pair fluents between entities first); `total: true` is refused until c3, and so are a `place_event_types` entry and a `locations` key. The initial pairs follow the specification's procedure in a fixed draw order: for a functional pair fluent one draw against the rate per entity in entity order, then the partner uniformly among the entities that keep the profile (an entity whose row was already filled by an earlier partner's draw, in a symmetric functional pair fluent, or with no eligible partner, gets none); otherwise one draw per unordered pair (`i < j`, row by row) for a symmetric or an asymmetric pair fluent, with one more draw (`rng.integers(2)`) for the direction of an asymmetric pair, and one draw per ordered pair (row by row) for a general one. A rate of 0 draws as the others do and selects nothing, so that c2b's draws are uniform whatever the rates. `initial_pairs.csv` has the columns `pair_fluent`, `first`, `second`: one row per ordered pair that holds, both directions of a symmetric pair, by pair fluent, then by first and second in entity order. `profile_violations(fluent, matrix)` is the one check of a profile, which the generator, the loader, and the runtime share. +4. **The definition.** The symbol kind `pair_fluent` (`label`, `kind`, `derived` false, `fluent` true, `arity` 2, `symmetry`, `functional`, `total`, `initial_rate`, all hashed) after the numeric fluents and before the event types, as the fluent-like symbols stand together. The literal table holds the entity-scope literals `pair_some` and `pair_someof` (role null, `pair_fluent`; keys `SOME.PAIRFL.k` and `SOMEOF.PAIRFL.k`) where a derived fluent's rule reads them, after the numeric thresholds in the group order, computed before the matrices from the state's pairs as the fluent and numeric threshold literals are. The pair literals of preconditions and conditions live in the event-type records with the same four kinds, as the Boolean and numeric precondition literals do (they are judged per binding outside the matrices); the loader accepts all four kinds in the table (the two-argument kinds with the role `binding` and the roles `first` and `second`), so that a definition that lists them is well formed. A precondition pair literal's record is `{kind, pair_fluent, first, second, value}` or `{kind, pair_fluent, role, value}`, with `origin: profile` for a compiled literal and no `origin` key otherwise; a Boolean or numeric literal has no `kind` key, so a reader tells them apart by it. A pair effect's record is `{kind: pair, pair_fluent, first, second, value, expression}`, with `condition` as any effect. A pair condition literal's record is the precondition form, its `kind` one of the four beside `feature`, `fluent`, and `numeric`. +5. **`PairLiteral` and `Effect.second`.** `dynamics.py` gains `PairLiteral(kind, pair_fluent, first, second, value, origin)`, whose `role` and `fluent` properties name the first role and the pair fluent so that code that counts literals by role (`world_stats`) reads it as the other kinds, and `Effect.second`, the second role of a pair effect (`Effect.pair`). The event-file parser replaces each pair literal's text with a placeholder name before the taxonomy's expression parser runs, because the parentheses and the comma of `PAIRFL.1(agent, patient)` are not in the expression grammar, and maps the placeholders back when it visits the parsed expression; `parse_condition` recognizes a pair literal by its form. +6. **What a pair literal names, and achievability.** For the rules of "Preconditions", "Effects", and "Conditions (F5)", a literal names a pair (both directions of a symmetric pair fluent are one pair), the row of a role (`SOME`; `SOMEOF` of a symmetric pair fluent names the same set), the column of a role (`SOMEOF`), or the rows of both roles (`SAME`). Two literals that name the same thing with different values contradict, and the same thing twice is a duplicate; two effects on one pair with different values contradict, and with the same value are a duplicate; an add whose precondition requires the pair, or a delete whose precondition requires its absence, changes nothing; a pair condition literal names no pair or row the precondition names (the rule read exactly: a condition that another literal entails without naming the same pair or row is allowed), nor the pair the effect itself writes (as a Boolean condition never names the fluent the effect sets: with the condition true the effect changes the pair, with it false the pair already has the value, so the condition would be a condition in name only). Achievable: a positive literal when its pair fluent's initial rate is above 0 or some effect adds a pair of it; a negative one when the rate is below 1 ("some initial state lacks it"), the pair fluent is functional (every other pair of a row is absent), or some effect deletes one; `SAME` on a symmetric functional pair fluent is never achievable, because two entities with one partner would give that partner two partners. For an explicit entry each rule is a `ConfigError` naming the file and the entry; `check_dynamics` checks the same rules on any definition and raises a `ValueError` naming the event type, as it does for the Boolean rules. +7. **Compiled literals.** `compile_profile_literals` appends the compiled literals (F4) to every event type with an unconditional add effect after the fix-ups and before the conditions are drawn, in effect order (`NOT PAIRFL.k(second, first)` for an asymmetric pair fluent, `NOT SOME.PAIRFL.k(first)` for a functional one, and `NOT SOME.PAIRFL.k(second)` too for a symmetric functional one), each with the origin `profile`, a literal already present by what it names (whatever its origin) not added twice. An explicit entry whose compiled literals contradict its precondition is an error naming the file and the entry, and a pair condition of an explicit entry is checked against the compiled literals as against the precondition. `redraw_preconditions` keeps the compiled literals of an event type whose own literals are drawn again, and `check_dynamics` requires every add to carry them. A conditional add on an asymmetric or functional pair fluent is refused in an event file (c2b redraws it). +8. **The runtime.** `State.pairs` is a tuple of Boolean matrices, one per pair fluent in symbol order, read-only, with an empty diagonal; `State.from_true` takes the pairs as `[pair fluent, first, second]` triples (the definition's initial pairs when None), and `pairs_record` gives them back in the order of `initial_pairs.csv`. A definition that carries no initial pairs (a fixture's, whose initial pairs come apart from the record) starts with none. `derive` fills the `pair_some` and `pair_someof` literals from the state's pairs before the entity layers; `pair_literal_holds` judges the four kinds over a bindings array (a symmetric pair fluent holds both directions of every pair, so a literal about it reads both without a special case). The facts an event reads and writes are keyed `(entity, fluent)` for a base fluent and `(pair fluent, first, second)` for an ordered pair: a pair effect writes its pair and, for a symmetric pair fluent, the reverse; a `pair` literal reads its pair (both directions when symmetric), `SOME` every pair of the row, `SOMEOF` every pair of the column, `SAME` every pair of both rows, rows and columns ranging over every entity of the definition (not only an episode's participants, which the runtime does not know), and a literal on a derived fluent reads, through `RuntimeDefinition.pair_cone`, the rows and columns of the entity its rule reads, directly or through other derived fluents, beside the base fluents of `cone`. `apply` writes the pairs of the fired effects into copies of the matrices and then checks every profile; a violation is `ProfileViolationError` (kind `profile`), naming the pair fluent, the pairs that hold, and the events. The error kinds of the fixtures stay `illegal` and `interference`. +9. **The `spatial` block.** The event file's block is checked for its form (a pair fluent of the file's `pair_fluents` block, a `detector` name, a `parameters` mapping of numbers or strings) and written last in `definition.json` as `spatial` with `spatial_id`, the SHA-256 of its canonical JSON; both are absent without a block, and neither is hashed into the rule-set identity. The loader checks the form and the identity (a wrong `spatial_id`, a block without its identity, an unknown pair fluent, or a malformed entry is a `DefinitionError`) and keeps both on `RuntimeDefinition`. +10. **Episodes and histories.** An episode's initial state holds the initial pairs among its participants and no other pair (WM.38: "an episode keeps the pairs among its participants"), so that a history's `initial_pairs` (every ordered pair that holds among the participants, both directions of a symmetric pair, as triples, after `initial_numeric`) is the whole pair part of the episode's initial state, and `replay` builds the state from it without the definition's initial pairs. Under `scene.initial: redraw` the pairs are drawn again among the participants, in entity order, by the initial-pair procedure at each pair fluent's rate, after the Boolean and numeric redraws, so the participants and the Boolean and numeric redraws of an episode are those of the same world without pair fluents. A pair change is `{"pair", "first", "second", "to"}`, one per pair effect whose fired write changed the value, in the direction the effect names (`pair_changes_of`); `replay` checks every event's pair changes against the same function, and refuses a history without `initial_pairs` on a world with pair fluents. A world without pair fluents writes neither key, and its histories are byte-identical to stage c1's. +11. **The evaluator and the fixtures.** `BruteForce` reads the pair part of a state as sets of label pairs (`BrutePairs`, set with `with_pairs` as the numeric part is with `with_numeric`), never through the matrices: the pair literals of rules, preconditions, and conditions, the pair effects (both directions of a symmetric pair), the read and write sets with rows and columns over every entity, and the profiles after a step (`check_profiles`). The fixture format gains `initial_pairs` after `initial_numeric` and `pairs` after `numeric` in each step, both lists of triples in the order of `initial_pairs.csv`; `check_fixture` and `check_fixture_brute` compare them; `generate_fixture` takes `initial_pairs`; the version stays 1. The hand world gains three pair fluents (`PAIRFL.1` symmetric, `PAIRFL.2` asymmetric, `PAIRFL.3` symmetric functional, every initial rate 0), a derived fluent `BOOLFL.4 = SOME.PAIRFL.1`, and five event types able for every ordered pair (`EVENTTYPE2.2`, a symmetric add; `EVENTTYPE2.3`, a delete that needs the pair; `EVENTTYPE2.4`, an asymmetric add with its compiled literal; `EVENTTYPE2.5`, a functional add with its two compiled literals; `EVENTTYPE2.6`, a pair condition); `hand_29` to `hand_37` are fixtures 1 to 4, 10, 12 to 14, and 16 of the specification. The earlier hand fixtures gained `initial_pairs: []`, `pairs: []` after each step, the new symbols and event types in their definitions, and the new event types' legal bindings; `test_quiescence_ends_an_episode` lists the new always-legal event types among those it removes. The tiny fixtures did not change (`make-fixtures` on `data/world/tiny.yaml` gives the committed files byte for byte). +12. **The corpus, the statistics, and the command line.** `load_world` refuses a world with pair fluents with a `CorpusError` that names stage c4. `world_stats.yaml` counts a pair condition literal under `condition_literals.by_symbol.pair` only when the world has one, so that a world without pair fluents writes its statistics as before; `absorbing_fluents` and `enabling_edges` leave pair effects out, and `episode_stats` counts Boolean changes only (the pair statistics are c2b's). The `define` line names the pair fluents only when there are any, so the snapshot's quick-start figure did not move. +13. **The example world.** `data/world/events/pair_example.yaml` defines four pair fluents (symmetric, asymmetric, general functional, and asymmetric functional with the rate 0, so one has no initial pairs) and five explicit event types over them with every kind of pair literal, adds and deletes, and two pair conditions (`SOME` and `pair`), each with the explicit requirement `TRUE`, which keeps the constraints of the features defining at the parent (WM.E106), so that every event type occurs in the statistics episodes (`EVENTTYPE2.2.1`, the `SAME` chain, in about one step in a thousand); `data/world/tiny_pairs.yaml` uses it with the tiny world, places kept. It is the world of the acceptance checks. +14. **A rounded zero is a positive zero** (the follow-up under item 1, WM.E269): `rounded_similarity` adds `0.0` after rounding, which turns `-0.0` into `0.0` and leaves every other value as it is; the rule is applied to the similarity range alone, because no configuration in `data/` writes a `-0.0` anywhere else. + +## File formats added + +- **`initial_pairs.csv`** (a world with pair fluents): columns `pair_fluent`, `first`, `second`; one row per ordered pair that holds at `TIME.1`, both directions of a symmetric pair, by pair fluent in label order, then by first and second in entity order. Read by `load_definition` beside `entities.csv` and `places.csv`, and required when the definition has pair fluents. +- **`definition.json`**: the symbol kind `pair_fluent`; the literal kinds `pair_some` and `pair_someof` (and, accepted, `pair` and `pair_same`); pair literals in `precondition.literals` and in `condition.literals` (`kind`, `pair_fluent`, `first` and `second` or `role`, `value`, `origin` when compiled); pair effects (`kind: pair`, `pair_fluent`, `first`, `second`, `value`, `expression`); `spatial` and `spatial_id` last, only with a block. +- **Histories**: `initial_pairs` after `initial_numeric` (a list of `[pair fluent, first, second]` triples, `[]` when none holds), and pair changes `{"pair", "first", "second", "to"}` among an event's `changes`; a world without pair fluents writes neither. +- **Fixtures**: `initial_pairs` after `initial_numeric`, and `pairs` after `numeric` in each step, in the form of the histories' `initial_pairs`; the version stays 1. `tests/fixtures/world/README.md` has a "Pair fluents (stage c2a)" section with the semantics operation by operation, and says that the total functional replacement comes in c3. +- **The event file**: the `pair_fluents` block (`PAIRFL.: {symmetry, functional, initial_rate}`), pair literals in `precondition` and in a `when ... then` condition, pair effects (`PAIRFL.(agent, patient) := 0 | 1`), and the `spatial` block (`PAIRFL.: {detector, parameters}`). +- **`config.yaml`** of every world: the seeds of `world:pair_fluents` and `world:pair_initial` under `provenance.stream_seeds`. + +## Changed expectations + +**Item 1** rewrote the golden hashes of `summary.yaml` for five taxonomy configurations (above). No snapshot figure moved, and no baseline: the rule-set identities of the tiny and default worlds and of every off setting are those of stage c1, as the determinism rule requires, because no world without an event file that defines pair fluents draws from the new streams. + +The hand-written fixtures `hand_01` to `hand_28` were rewritten by `python tests/world/hand_world.py` with the extended hand world (their rule-set identity moved with the new symbols and event types), and `hand_29` to `hand_37` were added. The tiny fixtures did not change. `tests/world/test_world_define.py` lists the two new streams, `tests/world/test_world_places.py` checks that the place streams come before the pair streams, and `tests/world/test_world_episodes.py` removes the new always-legal hand event types in its quiescence test. + +## Run times + +Measured on October 21, 2026, with nothing else running (the define time from a second run of the folder; the generate time from the regeneration of `runs/`): + +| Step | Stage c2a | Stage c1 | Change | +| --- | --- | --- | --- | +| `define data/world/default.yaml` | 6.8 s | 6.4 s | +6%: the default world has no pair fluents, so the stage adds the empty pair part of every state and the profile check of no pair fluent to the 1,000 statistics episodes; the rest is noise of the machine | +| `generate data/corpus/default.yaml` | 142 s | 134 s | +6%: the corpus does not read pair fluents, and the world is the same; noise of the machine (stage b3b measured 141 s on the same corpus) | + +No step grew by more than half. The tiny chain (world and corpus) runs in about 3 seconds; `define data/world/tiny_pairs.yaml` takes about 2 seconds. + +## Documents changed + +- `docs/DECISIONS.md`: WM.61 and WM.62 (Jon's rulings of October 21, 2026, after WM.60) and WM.E256 to WM.E268 (the engineering choices above, status "Working design", after WM.E255). Appended; no existing row edited. +- `docs/specs/WORLD_PHASE_C.md`: "(As built, stage c2a: ...)" notes on "Initial pairs between entities" (the event file's block, the draw order, the columns of `initial_pairs.csv`), "Literals over pairs" (the entity-scope kinds in the literal table, the records' four kinds, `SAME` on a symmetric functional pair fluent), "Compiled literals (F4)" (when they are appended, the explicit contradiction error, the redraws), "No contradictions, no empty effects" (the explicit errors, the effect record), "Conditions (F5)" (what a literal names, the exact reading, the written pair), "Derived fluents" (the rule inputs, `pair_cone`), "`apply`" (`State.pairs`, the replacement waiting for c3, no fixture ends in the profile error), "Non-interference" (the keys, rows over every entity), "The `spatial` block" (the form, the keys, the loader), "Initial state" of "Episodes" (the pairs among the participants, the redraw order), "Histories" (`initial_pairs`, the pair change), "Explicit event files" (the blocks, the errors, c3's entries refused), "Determinism" (the two streams), "Conformance fixtures" (`initial_pairs`, `pairs`, the hand fixtures), the stage c2a entry of "Build stages" (the proposal file and the WM.E rows), and the stage c3 entry (Jon's ruling WM.62 on the tiny place requirements). +- `docs/guides/WORLD.md`: the status, the "Pair fluents" concept, the labels, the streams, `define`, `check-fixtures` (43 fixtures), the outputs (`initial_pairs.csv`, the definition's symbol, literal, record, and `spatial` keys, the `pair` count of the statistics, the histories), the configuration ("Pair fluents in the event file", with the example and the errors), the fixtures, the Python interface, a recipe, troubleshooting, the run times, and the references. +- `docs/guides/TAXONOMY.md`: `summary.yaml`'s similarity range at 6 decimals (WM.61). +- `docs/guides/README.md`: the status table, the run times, and the chain paragraph's date. `docs/guides/CORPUS.md`: the run times. +- `CLAUDE.md`: the status of items 9 and 10 (stage c2a built, c2b next) and the `define data/world/tiny_pairs.yaml` command. +- `tests/fixtures/world/README.md`: the fixture keys `initial_pairs` and `pairs`, the pair fluent symbol's float, and a "Pair fluents (stage c2a)" section for the Rust runtime: the symbols, literals, records, compiled literals, literal values, the operations, non-interference over pairs, and the histories' additions, with the total functional replacement named as c3's. + +## Test results + +The full check list of `CLAUDE.md`, run on October 21, 2026, at the end of the stage: + +| Check | Result | +| --- | --- | +| `cargo fmt --all --check`, `cargo clippy --workspace --all-targets --all-features -- -D warnings`, `cargo test --workspace --all-features` | passed (the Rust crates are untouched by the stage) | +| `ruff check python tests` | passed | +| `pytest -n auto` (the fast tier) | passed in 94 s when run alone, nothing skipped | +| `pytest --all -n auto` (both tiers, 2,183 tests) | passed in 370 s, nothing skipped | + +The new tests: `tests/world/test_world_pairs.py` (the event file's pair fluents and the world's; the streams' order; the symbols and their hashing; the records with pair literals, effects, and compiled literals; `initial_pairs.csv` and the profiles of the initial pairs, the loader, and the draw procedure for every profile and rate; a world without pair fluents writes none of the new keys; the acceptance checks of `legal` and `derive` against the brute-force evaluator in 1,000 random states, of applying together against one at a time in every order on 1,000 random non-interfering sets, of interference against the evaluator with the error's wording, of no profile broken in 1,000 episodes under `keep` and under `redraw` with replay of every pair change, and of the histories' form; the symmetric writes; the profile invariant on a broken definition; the illegal-event message; the redraw after every other draw; `simulate`; twenty broken two-place entries, each naming the file and the entry, with the one-place rule and the block's validation apart; the `spatial` block; the generated fixtures of the pair world through both checkers; the loader's validation of pair records; the corpus's refusal; the statistics; and the hand fixtures), and the nine hand fixtures through `test_world_fixtures.py`. The golden test of the taxonomy compares the rounded range. Every acceptance check of phases (a) and (b) and of stage c1 still passes: the tests of stages a1 to c1 run unchanged in their meaning, with the three edits listed under "Changed expectations". + +## Open questions + +None of these changes the design; each is recorded for Jon's eye. + +1. **The literal table and the four kinds.** "Literals over pairs" says the four kinds join the literal table. As built, the table holds the two entity-scope kinds (`SOME` and `SOMEOF` of the entity itself, which rules read), and the records of preconditions and conditions hold all four, as they hold the Boolean and numeric literals; the loader accepts all four in the table. If the Rust runtime is better served by every pair literal in the table (with the role `binding` and the two roles), the builder can add them without changing the records, at the cost of an identity change for worlds with pair fluents. +2. **The rule that a condition names no pair or row the precondition names** is read exactly. Under a symmetric pair fluent a condition can be entailed by the precondition without naming the same thing (`PAIRFL.1(agent, patient)` in the precondition and `SOMEOF.PAIRFL.1(patient)` in the condition: the condition is always true), and is allowed. An entailment check would be a larger rule; c2b's draws could redraw such a condition by the same exact rule, or by a check of what the precondition entails. +3. **`SAME` on a symmetric functional pair fluent** is never satisfiable (two entities with one partner give that partner two), so it is refused as unachievable. The specification does not say so; c2b's draws should never draw it, and the default profiles (`functional_rate` 0.25, equal symmetries) would draw a symmetric functional pair fluent for about one pair fluent in twelve. +4. **A pair condition on the pair the effect writes** is refused, by analogy with the Boolean rule ("the condition never names the fluent the effect sets"). The specification lists the rule for Boolean conditions only. +5. **Negative achievability** is read as an initial rate below 1, a functional pair fluent, or some delete. A general pair fluent with the rate 1 and no delete has every pair at `TIME.1` and forever, so a negative literal on it is never true; an asymmetric one with the rate 1 has one direction of every pair, so a negative literal on it is true half the time, which the reading counts as achievable only through the rate (1 is not below 1): such a literal is refused. A rate of 1 is an odd setting; the reading errs on the side of refusing. +6. **Rows and columns range over every entity of the definition**, not only the participants of an episode, for non-interference. Within an episode only pairs among participants can be written, so the two readings give the same verdict; the wider one costs nothing and needs no knowledge of the episode in the runtime. +7. **The redraw draws the participants' pairs in entity order**, not in the order the participants were drawn (the Boolean and numeric redraws go in drawn order). Either order is deterministic; entity order matches the procedure of the initial pairs, which is defined over the entities in order. +8. **A history's `initial_pairs` lists both directions of a symmetric pair.** The specification's example shows one triple for `PAIRFL.3`, whose profile it does not give. Listing every ordered pair that holds makes the list the pair part of the state, as `initial` lists every true fluent; the changes record a symmetric pair once, as the specification says. +9. **The taxonomy's scalar values at full precision** in `entities.csv` and `places.csv` (c1's open question 1) remain as they were: the check of item 1 covered the YAML and JSON outputs, which Jon's Linux run compared, and found nothing else. +10. **An explicit requirement `TRUE` keeps the parent's constraints** (WM.E106), so the example world's "always able" event types are able for the pairs of their parent's base relation. The comment of `pair_example.yaml` says so. A world in which every pair is able would need a parent without defining features, which no event file can give. +11. **The corpus's refusal and the default configurations of c2b.** The refusal (item 12 of the stage) is right for this stage: no corpus states a fact its oracle cannot check. But "Configuration" of the specification gives `data/world/default.yaml` 4 pair fluents and `data/world/tiny.yaml` 2 in stage c2b, and the tiny and default corpora name those worlds, so from c2b until c4 (which extends the oracle to pairs) the corpora could not run on their own worlds unless c2b keeps the pair settings out of `tiny.yaml` and `default.yaml` (in configurations of their own, as this stage's `tiny_pairs.yaml`), or the corpus of c2b and c3 runs on a world whose pair fluents it ignores. The second would state nothing false (the corpus reads no pair), but its scenes would hold pair changes its oracle does not replay. This is a question for the c2b session's prompt, not a change to this stage. + +## Notes for stage c2b + +What the builder of the generation of pair fluents (drawn profiles, initial rates, pair literals in preconditions, pair effects, `pair` conditions, derived rules over `SOME` and `SOMEOF`, the quotas, and the statistics) must know: + +1. **The configuration block `pair_fluents`** of the specification (`count`, `symmetry`, `functional_rate`, `initial_rates`, `literal_kinds`, `same_rules`, `derived_rules`, `rule_weight`, `fluents`) and the shares `event_types.preconditions.pair_share`, `event_types.effects.pair_share`, and `condition_kinds.pair` are not read yet: `config.py` is untouched. `generate_pair_fluents(entity_count, streams, explicit, explicit_source)` in `pair_fluents.py` builds the pair fluents from the event file's block alone; c2b draws `count` profiles and rates from `streams.pair_fluents` first, in label order, and lets a `fluents.PAIRFL.` override or an event file entry replace a draw (the event file's labels must then be among `PAIRFL.1` to `PAIRFL.count`, as `fluents.BOOLFL.` must be among the fluents). The initial pairs already come from `streams.pair_initial` by the specification's procedure (`draw_initial_pairs`), and a rate of 0 draws as the others do. +2. **The dynamics.** `_Dynamics` in `event_types.py` takes `pair_fluents`; `produced()` and `achievable()` leave pair effects out, `literal_achievable` and `_condition_literal_achievable` judge a `PairLiteral` by `pair_literal_achievable(literal, fluent, adds, deletes)` (`pair_adds_and_deletes()` gives the sets), `_literal_slot` keys a precondition literal by what it names, `condition_keys` does not cover pair literals (a Boolean literal can never name a pair fluent), and `compile_profile_literals` runs for every draft after the fix-ups: c2b's drawn pair literals and effects go into `draw_preconditions` and `draw_effects` as replacements at the shares from `world:pair_preconditions` and `world:pair_effects` (the pattern of the numeric streams, WM.E193), before `compile_profile_literals`; a drawn add whose compiled literals contradict the precondition, a contradiction, an empty effect, an unachievable literal, and a conditional add on an asymmetric or functional pair fluent are redrawn where the explicit checks raise. The checks of an explicit entry are `_check_explicit_pair_literal`, `_check_explicit_pair_effect`, and `_check_pair_condition_against`; `check_dynamics` (`_check_pair_effect`, `_check_pair_condition_literal`) is the invariant checker for both. `redraw_preconditions` keeps compiled literals; a redrawn own literal could be a pair literal in c2b, drawn from a part of `world:pair_preconditions`. +3. **Derived rules.** A rule reads `SOME.PAIRFL.k` or `SOMEOF.PAIRFL.k` as an input key; `build_definition` turns it into a `pair_literal_spec` and `pair_literal_key_of_rule_input` recognizes it. `fluents._draw_rules` and `_inputs` do not offer pair literals in the pool yet: c2b adds them with the weight `pair_fluents.rule_weight`, the quota `derived_rules` first, and the values for `derived_initial_values` from the initial pairs (`PairFluents.initial_pairs`), which `entity_literal_values` already takes (`pairs=`). The runtime's `derive`, `pair_cone`, and the evaluator's `pair_cone` already read such rules; the hand world's `BOOLFL.4` exercises them. +4. **The statistics.** `world_stats.yaml` has no `pair_fluents` block yet; `world_stats` counts a pair condition literal under `by_symbol.pair` only when there is one, and `episode_stats` counts Boolean changes alone. The specification's block (pair fluents by profile, initial rates and pairs, pair literals by kind and role, compiled literals by origin, pair effects by value, pair conditions, and the episodes' pair changes per event) is c2b's, as is `figures.py`'s reading of it and the off settings of "Determinism" (`pair_fluents: {count: 0, locations: 0}`, and the pair shares at 0) with their baselines in the snapshot. +5. **The fixtures.** `fixture_initial_pairs(definition)` gives a built definition's initial pairs; `world_fixtures` passes them, so `make-fixtures` on a world with pair fluents already writes `initial_pairs` and `pairs`. The tiny world's committed fixtures gain pair fluents when `data/world/tiny.yaml` gets its `pair_fluents` block in c2b (the specification: 2 pair fluents between entities, 1 location fluent in c3). +6. **The state's pair part** is a tuple of `(entities, entities)` matrices; `profile_violations` handles a non-square matrix for the total profile already, and `_check_state` checks the square shape, which c3 relaxes for a location fluent's `(entities, places)`. +7. **Labels.** `PairFluent.index` is the number in the label; `pair_label(i)` makes it. The pair fluents between entities are numbered first (WM.54), so c3's location fluents take the numbers after `count`. +8. **The corpus** refuses a world with pair fluents in `load_world`; the refusal stays until c4 extends the oracle's replay, and the tiny corpus's world must keep no pair fluents until then, or c2b's `tiny.yaml` change breaks the corpus tests. Either c2b leaves `tiny.yaml` without pair fluents and gives the pair settings to a `tiny_pairs.yaml` of its own, or c4 comes first for the corpus; this stage's `data/world/tiny_pairs.yaml` is the natural home of the tiny pair settings until c4. diff --git a/docs/specs/WORLD_PHASE_C.md b/docs/specs/WORLD_PHASE_C.md index 6488246..1195fc2 100644 --- a/docs/specs/WORLD_PHASE_C.md +++ b/docs/specs/WORLD_PHASE_C.md @@ -59,7 +59,7 @@ New labels, in the style of "Labels" in `WORLD_AND_LANGUAGE.md`: - **How many.** `pair_fluents.count` pair fluents between entities (default 4), numbered first, `PAIRFL.1` to `PAIRFL.n`, then `pair_fluents.locations` location fluents (default 1), so the default world's location fluent is `PAIRFL.5`. Turning places on or off leaves the labels of the pair fluents between entities unchanged (review point 6). With `places: null`, `locations` must be 0. - **Profiles.** A location fluent's profile is fixed: `general`, functional, and total, from entities to places. A pair fluent between entities draws its symmetry from `pair_fluents.symmetry` (default equal weights over `symmetric`, `asymmetric`, and `general`), and is functional with probability `pair_fluents.functional_rate` (default 0.25). It is never total (ruling 2a). The draws come from the stream `world:pair_fluents`. - **What a profile means.** Symmetric: `(x, y)` holds exactly when `(y, x)` holds. Asymmetric: never both. Functional: at most one pair in each row; for a symmetric functional pair fluent, at most one partner in each row and each column. Total: exactly one pair in each row of every entity. -- **Initial pairs between entities.** Each pair fluent between entities gets an initial rate, drawn uniformly from `pair_fluents.initial_rates` (default `[0.0, 0.05, 0.2]`). Pairs are drawn once, when the world is generated, from the stream `world:pair_initial`, respecting the profile: a symmetric pair fluent draws each unordered pair at the rate and writes both directions; an asymmetric one draws each unordered pair at the rate and then one direction; a general one draws each ordered pair at the rate; a functional one gives each entity, at the rate, one partner drawn uniformly among the entities that still keep the profile. The pairs are written to `initial_pairs.csv`. +- **Initial pairs between entities.** Each pair fluent between entities gets an initial rate, drawn uniformly from `pair_fluents.initial_rates` (default `[0.0, 0.05, 0.2]`). Pairs are drawn once, when the world is generated, from the stream `world:pair_initial`, respecting the profile: a symmetric pair fluent draws each unordered pair at the rate and writes both directions; an asymmetric one draws each unordered pair at the rate and then one direction; a general one draws each ordered pair at the rate; a functional one gives each entity, at the rate, one partner drawn uniformly among the entities that still keep the profile. The pairs are written to `initial_pairs.csv`. (As built, stage c2a: in c2a the pair fluents and their rates come from the event file's `pair_fluents` block, labeled `PAIRFL.1` and on without a gap; the draws of `world:pair_initial` are made fluent by fluent in a fixed order, per entity for a functional pair fluent and then the partner among the entities that keep the profile, per unordered pair with a direction draw for an asymmetric one, per ordered pair for a general one, a rate of 0 drawing as the others and selecting nothing; `initial_pairs.csv` has the columns `pair_fluent`, `first`, `second`, one row per ordered pair that holds, both directions of a symmetric pair.) - **Initial places (habitats).** Each entity's place at `TIME.1` is drawn once, when the world is generated, from the stream `world:habitats`. The weight of place `q` for entity `x` is `(1 + a(x, q)) ^ places.habitat_strength`, where `a(x, q)` is the number of place event types whose requirement holds for the binding `(x, q)`. The default strength is 1; 0 gives a uniform draw. With several location fluents, each draws its own place in label order. The places are columns of `entities.csv`, one per location fluent, named by the fluent's label. - **Per pair fluent overrides.** `pair_fluents.fluents.PAIRFL.: {symmetry, functional, initial_rate}` replaces the draws for one pair fluent between entities. @@ -84,28 +84,28 @@ New labels, in the style of "Labels" in `WORLD_AND_LANGUAGE.md`: - `SOME` and `SOMEOF` of a location fluent are excluded where they are constant: `SOME` of an entity on a location fluent is always true, so it is never drawn. `SOMEOF` of a place on a location fluent ("someone is at the place") can be drawn in the precondition of a place event type. - `x` and `y` are roles of a binding in a precondition or a condition, and an entity in a derived fluent's rule. A literal about a symmetric pair fluent reads both directions of its pair. -- The literals are computed before the matrices, as threshold literals are, and join the literal table of `definition.json` with the kinds `pair`, `pair_some`, `pair_someof`, and `pair_same`. +- The literals are computed before the matrices, as threshold literals are, and join the literal table of `definition.json` with the kinds `pair`, `pair_some`, `pair_someof`, and `pair_same`. (As built, stage c2a: the literal table holds the entity-scope literals `pair_some` and `pair_someof` (role null, keys `SOME.PAIRFL.k` and `SOMEOF.PAIRFL.k`) where a derived fluent's rule reads them, after the numeric thresholds; the pair literals of preconditions and conditions live in the event-type records with the same four kinds, as the Boolean and numeric literals do, and the loader accepts all four kinds in the table. In the records a pair literal is `{kind, pair_fluent, first, second}` or `{kind, pair_fluent, role}` with its `value`, and `origin` for a compiled literal. `SAME` on a symmetric functional pair fluent is never achievable, because two entities with one partner would give that partner two, so an event file may not read it positively.) ### Preconditions - **Pair literals.** A share `event_types.preconditions.pair_share` (default 0.2) of precondition literals is a pair literal when the world has pair fluents. As for numeric literals (WM.E193), each literal is drawn as before and then replaced at the share, from the stream `world:pair_preconditions`, so with the share at 0 the preconditions are those of the world without pair fluents. The kind of pair literal is drawn from `pair_fluents.literal_kinds` (default `{pair: 0.6, same: 0.1, some: 0.3}`, where `some` covers `SOME` and `SOMEOF`), its pair fluent uniformly among those the kind allows, its direction or role uniformly, and its polarity uniformly. A quota comes first: `pair_fluents.same_rules` two-place event types (default 2) gain the precondition literal `SAME.PAIRFL.k(agent, patient)` on a location fluent, drawn from their own part of `world:pair_preconditions` among two-place event types for which some able binding can satisfy the rest of the precondition; with no location fluents, the quota does nothing (review point 4). One-place event types draw only `some` literals on the agent. Place event types draw `pair` and `same` literals only on location fluents, and `some` literals on the agent's pair fluents between entities or as `SOMEOF` of the place. - **Achievable.** A positive pair literal is achievable when its pair fluent has a positive initial rate or some effect adds the pair; a negative one, when some initial state lacks it or some effect deletes it. `SAME` on a location fluent is always achievable, because entities can share a place. The generator redraws a literal that is not achievable. -- **Compiled literals (F4).** After the preconditions and effects are drawn, the generator adds, for each add effect: `NOT PAIRFL.k(patient, agent)` for an asymmetric pair fluent (or the same with the roles exchanged for an add in the other direction); `NOT SOME.PAIRFL.k(first)` for a functional one, and also `NOT SOME.PAIRFL.k(second)` for a symmetric functional one; and `NOT PAIRFL.k(agent, patient)` for a move. A compiled literal already present is not added twice. An event type whose compiled literals contradict its own precondition can never happen: its effect is redrawn. Compiled literals are written in `definition.json` with their origin, and are part of the rule-set identity. +- **Compiled literals (F4).** After the preconditions and effects are drawn, the generator adds, for each add effect: `NOT PAIRFL.k(patient, agent)` for an asymmetric pair fluent (or the same with the roles exchanged for an add in the other direction); `NOT SOME.PAIRFL.k(first)` for a functional one, and also `NOT SOME.PAIRFL.k(second)` for a symmetric functional one; and `NOT PAIRFL.k(agent, patient)` for a move. A compiled literal already present is not added twice. An event type whose compiled literals contradict its own precondition can never happen: its effect is redrawn. Compiled literals are written in `definition.json` with their origin, and are part of the rule-set identity. (As built, stage c2a: the compiled literals are appended to the precondition after the fix-ups and before the conditions are drawn, in effect order, and kept through the redraws of never-legal event types; an explicit entry whose compiled literals contradict its precondition is an error naming the file and the entry; `check_dynamics` requires every add to carry them.) - **Redraws of never-legal event types** (WM.E111) include place event types. ### Effects - **Pair effects.** An effect can add or delete a pair between the roles of a two-place event type: `PAIRFL.k(agent, patient) := 1` or `PAIRFL.k(patient, agent) := 0`. A share `event_types.effects.pair_share` (default 0.2) of the effects of two-place event types is a pair effect when the world has pair fluents between entities, drawn as before and then replaced at the share, from the stream `world:pair_effects`. The pair fluent, direction, and value are drawn uniformly. Pair effects attach to event-type features and to event types, as other effects do. - **No delete on a total functional pair fluent.** The generator never draws one, and an event file that writes one is a validation error. -- **No contradictions, no empty effects.** Two effects on the same pair (counting both directions of a symmetric pair fluent) with different values are a contradiction, resolved as in phase (a). An add whose precondition already requires the pair, or a delete whose precondition already requires its absence, is redrawn. +- **No contradictions, no empty effects.** Two effects on the same pair (counting both directions of a symmetric pair fluent) with different values are a contradiction, resolved as in phase (a). An add whose precondition already requires the pair, or a delete whose precondition already requires its absence, is redrawn. (As built, stage c2a: for explicit entries each is an error naming the file and the entry, a second effect on the same pair with the same value included; a pair effect is an `Effect` whose `second` names the second role, written as `{kind: pair, pair_fluent, first, second, value, expression}`.) ### Conditions (F5) -The kind `pair` joins `event_types.effects.condition_kinds`, default `{feature: 0.4, fluent: 0.4, numeric: 0.2, pair: 0.2}`. A pair condition literal is a `pair`, `same`, or `some` literal over the binding, drawn as for preconditions, from the stream `world:pair_conditions`. It must be achievable, and it names no pair or row that the precondition names. A conditional add on an asymmetric or functional pair fluent is redrawn, because its compiled literals would not be a plain precondition (F4). With the `pair` weight at 0, or no pair fluents, the conditions are drawn as in stage b3b. +The kind `pair` joins `event_types.effects.condition_kinds`, default `{feature: 0.4, fluent: 0.4, numeric: 0.2, pair: 0.2}`. A pair condition literal is a `pair`, `same`, or `some` literal over the binding, drawn as for preconditions, from the stream `world:pair_conditions`. It must be achievable, and it names no pair or row that the precondition names. A conditional add on an asymmetric or functional pair fluent is redrawn, because its compiled literals would not be a plain precondition (F4). With the `pair` weight at 0, or no pair fluents, the conditions are drawn as in stage b3b. (As built, stage c2a: what a literal names is the pair (one pair for both directions of a symmetric pair fluent), the row of a role for `SOME` (and for `SOMEOF` of a symmetric pair fluent), the column for `SOMEOF`, and the rows of both roles for `SAME`; the rule is read exactly, so a condition that another literal entails without naming the same pair or row is allowed. A pair condition never names the pair the effect itself writes, as a Boolean condition never names the fluent the effect sets. In the records a pair condition literal has one of the four pair kinds beside `feature`, `fluent`, and `numeric`.) ### Derived fluents -A derived Boolean fluent's rule may read `SOME.PAIRFL.k(x)` and `SOMEOF.PAIRFL.k(x)` of its own entity, on pair fluents between entities. The first `pair_fluents.derived_rules` derived fluents (default 1) must read at least one of them: their rules are drawn first, each with at least one such literal among its inputs. The other derived fluents draw from the input pool as before, where the literals weigh `pair_fluents.rule_weight` (default 1). With `pair_fluents.count: 0`, the quota does nothing, and the pool and the rules are unchanged (review point 3). A derived fluent's cone holds the rows and columns its literals read. +A derived Boolean fluent's rule may read `SOME.PAIRFL.k(x)` and `SOMEOF.PAIRFL.k(x)` of its own entity, on pair fluents between entities. The first `pair_fluents.derived_rules` derived fluents (default 1) must read at least one of them: their rules are drawn first, each with at least one such literal among its inputs. The other derived fluents draw from the input pool as before, where the literals weigh `pair_fluents.rule_weight` (default 1). With `pair_fluents.count: 0`, the quota does nothing, and the pool and the rules are unchanged (review point 3). A derived fluent's cone holds the rows and columns its literals read. (As built, stage c2a: a rule reads them as the inputs `SOME.PAIRFL.k` and `SOMEOF.PAIRFL.k`, literals of the kinds `pair_some` and `pair_someof` in the table; `RuntimeDefinition.pair_cone` gives the rows and columns a fluent reads through them, directly or through other derived fluents, which non-interference counts beside the base fluents of `cone`. The drawn rules come in c2b; the hand world's `BOOLFL.4 = SOME.PAIRFL.1` exercises the machinery.) ### The runtime @@ -113,18 +113,18 @@ A derived Boolean fluent's rule may read `SOME.PAIRFL.k(x)` and `SOMEOF.PAIRFL.k - **`derive`** computes the pair literals before the matrices, and the derived fluents from them. - **`able`** is unchanged. For a place event type, a binding is an entity and a place, and the requirement reads both. - **`legal`** is unchanged: the requirement and the precondition, compiled literals included. -- **`apply`.** Each effect whose condition holds is applied to the state at `TIME.k`. An add or delete of a symmetric pair writes both directions. An add in a total functional pair fluent deletes the old pair of the row, in the same step. After the step, every profile must hold: a violation raises `ProfileViolationError` (kind `profile`), naming the pair fluent, the pairs, and the events. With a correct definition, the error never happens, because the compiled literals make it impossible. -- **Non-interference.** An event reads the pairs, rows, and columns its precondition and conditions read, through the cones of derived fluents. It writes the pairs its effects write: both directions of a symmetric pair, and, for an add in a total functional pair fluent, the whole row. Two events interfere when one writes a pair that the other writes or reads, counting a row or a column as every pair in it. The rule of phase (a) is otherwise unchanged, so applying a step's events together equals applying them one at a time in any order. +- **`apply`.** Each effect whose condition holds is applied to the state at `TIME.k`. An add or delete of a symmetric pair writes both directions. An add in a total functional pair fluent deletes the old pair of the row, in the same step. After the step, every profile must hold: a violation raises `ProfileViolationError` (kind `profile`), naming the pair fluent, the pairs, and the events. With a correct definition, the error never happens, because the compiled literals make it impossible. (As built, stage c2a: `State.pairs` is a tuple of Boolean matrices, one per pair fluent in symbol order, with the entities as rows and the fluent's targets as columns, so that c3's location fluents add the second kind of target; the total functional replacement comes with them. No fixture ends in the profile error.) +- **Non-interference.** An event reads the pairs, rows, and columns its precondition and conditions read, through the cones of derived fluents. It writes the pairs its effects write: both directions of a symmetric pair, and, for an add in a total functional pair fluent, the whole row. Two events interfere when one writes a pair that the other writes or reads, counting a row or a column as every pair in it. The rule of phase (a) is otherwise unchanged, so applying a step's events together equals applying them one at a time in any order. (As built, stage c2a: a fact is keyed `(entity, fluent)` or `(pair fluent, first, second)`; a row or a column ranges over every entity of the definition, not only the participants of an episode; a literal about a symmetric pair fluent reads both directions of its pair, and a `SAME` literal both rows.) - **Places in the runtime.** Places are indexed apart from entities. A binding of a place event type is an entity index and a place index. `legal_bindings` lists them by agent, then place, in index order. (As built, stage c1: `RuntimeDefinition` loads `places.csv` beside `entities.csv`, `legal_bindings(definition, state, event_type, entities, places)` takes the places as a second list, every place when none, and `check_definition_agreement` covers the place scopes.) ### The `spatial` block (F6) -`definition.json` may hold a `spatial` block: for a pair fluent, a detector name and its parameters, as `{"PAIRFL.2": {"detector": "within", "parameters": {"distance": 1.0}}}`. `spatial_id` is the SHA-256 hash of the block's canonical JSON. Neither is part of the rule-set identity. The Python runtime checks the block's form only, and otherwise ignores it. Generated worlds have no block. An event file may give one. When the block is absent, `definition.json` has neither key. +`definition.json` may hold a `spatial` block: for a pair fluent, a detector name and its parameters, as `{"PAIRFL.2": {"detector": "within", "parameters": {"distance": 1.0}}}`. `spatial_id` is the SHA-256 hash of the block's canonical JSON. Neither is part of the rule-set identity. The Python runtime checks the block's form only, and otherwise ignores it. Generated worlds have no block. An event file may give one. When the block is absent, `definition.json` has neither key. (As built, stage c2a: the block names a pair fluent of the event file, with a `detector` name and a `parameters` mapping of numbers or strings; `spatial` and `spatial_id` are the last two keys of `definition.json`, and the loader checks the identity.) ### Episodes - **Places of an episode (F2).** After the participants are drawn, the episode's places are the distinct places of its participants at `TIME.1`, plus `scene.extra_places` (default 1) places drawn uniformly from the others. `scene.places: all` puts every place in every episode. A place event type's bindings range over the episode's places. -- **Initial state.** Each participant's place comes from `entities.csv`, and the pairs among the participants from `initial_pairs.csv`. With `scene.initial: redraw`, the episode also redraws its participants' places with the habitat weights and their pairs at each pair fluent's rate, from the episode's own part of the stream. With `scene.initial_location: seed`, every participant starts at the seed entity's place, and the episode's places are that place and the extra places. +- **Initial state.** Each participant's place comes from `entities.csv`, and the pairs among the participants from `initial_pairs.csv`. With `scene.initial: redraw`, the episode also redraws its participants' places with the habitat weights and their pairs at each pair fluent's rate, from the episode's own part of the stream. With `scene.initial_location: seed`, every participant starts at the seed entity's place, and the episode's places are that place and the extra places. (As built, stage c2a: an episode's initial state holds the initial pairs among its participants and no other pair, WM.38; the redraw draws the participants' pairs among themselves, in entity order, by the procedure of the initial pairs.) - **Order of draws.** The new draws of an episode (redrawn places and pairs, then extra places) come after every existing draw of the episode's part, so episodes of a world without pair fluents and places are unchanged. - **Selection.** `uniform_event` and `uniform_event_type` treat place event types as two-place event types: under `uniform_event_type`, `scene.transitive_share` covers both, and the event type is drawn among all event types with a patient. @@ -144,7 +144,7 @@ A derived Boolean fluent's rule may read `SOME.PAIRFL.k(x)` and `SOMEOF.PAIRFL.k ``` - `places`, `initial_places`, and `initial_pairs` appear only in a world with places or pair fluents. -- A pair change records the pair fluent, its two arguments, and the new value. A move records the deletion of the old place and the add of the new one, both as changes of the event. A symmetric pair is recorded once, in the direction the effect names, and replay writes both directions. +- A pair change records the pair fluent, its two arguments, and the new value. A move records the deletion of the old place and the add of the new one, both as changes of the event. A symmetric pair is recorded once, in the direction the effect names, and replay writes both directions. (As built, stage c2a: `initial_pairs` lists every ordered pair that holds among the participants, both directions of a symmetric pair, as `[pair fluent, first, second]` triples, after `initial_numeric`, and is the whole pair part of the episode's initial state; a pair change is `{"pair", "first", "second", "to"}`.) - `replay` reproduces every recorded pair change, and checks every profile at every time point. ### World statistics @@ -177,7 +177,7 @@ spatial: PAIRFL.2: {detector: within, parameters: {distance: 1.0}} ``` -Compiled literals are added to explicit entries too. An entry that breaks a rule (a delete on a total functional pair fluent, a `SAME` on a pair fluent that is not functional, a literal that is not achievable, an act without its location literal) is an error that names the file and the entry. `data/world/events/place_chain_example.yaml` builds the chain of go, approach, and eat. +Compiled literals are added to explicit entries too. An entry that breaks a rule (a delete on a total functional pair fluent, a `SAME` on a pair fluent that is not functional, a literal that is not achievable, an act without its location literal) is an error that names the file and the entry. `data/world/events/place_chain_example.yaml` builds the chain of go, approach, and eat. (As built, stage c2a: the `pair_fluents` block and the `spatial` block, pair literals in preconditions and conditions, and pair effects; `data/world/events/pair_example.yaml` with `data/world/tiny_pairs.yaml` is the example over four pair fluents. A location fluent (`total: true`), a `place_event_types` entry, and a `locations` key are errors until stage c3. The errors of an entry are those of "Preconditions", "Effects", and "Conditions (F5)", each naming the file and the entry.) ### Configuration @@ -222,13 +222,13 @@ Corpus configuration, under `scene`: `extra_places` (default 1), `places: episod ## Determinism - New world streams, created after the existing ones: `world:places`, `world:pair_fluents`, `world:pair_initial`, `world:place_event_types`, `world:place_dynamics`, `world:habitats`, `world:pair_preconditions`, `world:pair_effects`, and `world:pair_conditions`. -- **Off settings, each tested against an identity in the snapshot (PR.2):** with `places: null` and `pair_fluents: {count: 0, locations: 0}`, the world run is byte-identical to the world of stage b3b; with pair fluents but `pair_share: 0` for preconditions and effects and the `pair` condition weight at 0, the preconditions, effects, and conditions of the existing event types are those of the world without pair fluents; with `labels.kind: entity`, every taxonomy run is byte-identical to the run before the stage. (As built, stage c1: the world of stage b3b is read as the world after the rounding of drawn reals, Jon's ruling WM.60, which changed the tiny and default worlds once before places were built; the snapshot's baseline `places_off` holds the identities.) +- **Off settings, each tested against an identity in the snapshot (PR.2):** with `places: null` and `pair_fluents: {count: 0, locations: 0}`, the world run is byte-identical to the world of stage b3b; with pair fluents but `pair_share: 0` for preconditions and effects and the `pair` condition weight at 0, the preconditions, effects, and conditions of the existing event types are those of the world without pair fluents; with `labels.kind: entity`, every taxonomy run is byte-identical to the run before the stage. (As built, stage c1: the world of stage b3b is read as the world after the rounding of drawn reals, Jon's ruling WM.60, which changed the tiny and default worlds once before places were built; the snapshot's baseline `places_off` holds the identities.) (As built, stage c2a: `world:pair_fluents` and `world:pair_initial` exist after `world:place_event_types`; a world without an event file that defines pair fluents draws from neither, and its definition and identity are those of stage c1, so the snapshot's identities and baselines did not move.) - The rules of derived fluents may differ when pair fluents are present, because the input pool is larger, as with numeric fluents (stage b2). - The corpus makes no promise of byte identity (PR.2). Its new content draws from streams of its own where that is easy. ## Conformance fixtures -The fixture format gains `pair_fluents` and `places` in the definition, `initial_pairs` and `initial_places` beside `initial`, and, in each step, `pairs` (the pairs that hold after the step). The format's version stays 1, because the keys are absent from a world without them. `tests/fixtures/world/README.md` gives the semantics of pair literals, compiled literals, the total functional replacement, and the read and write sets for non-interference, operation by operation, for the Rust runtime. (As built, stage c1: a fixture of a world with places carries the places' static facts under `places`, after `entities`, and its inline definition the place symbols and scopes; a step's `legal` lists the performable event types only.) +The fixture format gains `pair_fluents` and `places` in the definition, `initial_pairs` and `initial_places` beside `initial`, and, in each step, `pairs` (the pairs that hold after the step). The format's version stays 1, because the keys are absent from a world without them. `tests/fixtures/world/README.md` gives the semantics of pair literals, compiled literals, the total functional replacement, and the read and write sets for non-interference, operation by operation, for the Rust runtime. (As built, stage c1: a fixture of a world with places carries the places' static facts under `places`, after `entities`, and its inline definition the place symbols and scopes; a step's `legal` lists the performable event types only.) (As built, stage c2a: `initial_pairs` after `initial_numeric` and `pairs` after `numeric` in each step, both lists of `[pair fluent, first, second]` triples in the order of `initial_pairs.csv`; `initial_places` waits for c3; the hand world has three pair fluents between entities and `hand_29` to `hand_37` are fixtures 1 to 4, 10, 12 to 14, and 16 below; the generated fixtures of a world with pair fluents come in c2b, and the tiny fixtures did not change.) Hand-written fixtures, over the hand world extended with two places, a location fluent, and three pair fluents between entities (symmetric, asymmetric, symmetric functional): @@ -327,9 +327,9 @@ A sentence with a prepositional predicate about a time point has the reading `st Each stage is one Claude Code session on a branch of its own (`world-c1`, and so on), on the staged track: a proposal file, WM.E rows, reference runs, the full check list, run times in the guides with growth of more than half flagged, figures through the snapshot and `python -m semantic_world.figures guides`, and the guides updated where the stage changes what they describe. Every acceptance criterion checks a property, never a count. 1. **c1. Places.** The taxonomy's `labels.kind`, the place taxonomy configurations, the second taxonomy run in the world, `places/` in the world run, `places.csv`, place comparison classes and `EVERYPLACE`, place event types with their requirements, capacities (`CAN.` and `CANBE.PLACEEVENTTYPE`), relation statistics, views with place columns (`view --places`), the `places` block of the statistics, and `WORLD.md`. No dynamics yet: place event types have no preconditions or effects in this stage, and episodes do not use places. *Accept:* with `places: null` the world is byte-identical to stage b3b (the snapshot identity); the place run is byte-identical to the taxonomy run alone with `labels.kind: place`; the matrices agree with the truth tables for every place rule and every place event type's requirement over every entity and place; capacities agree with a brute-force check; comparison classes of places agree with a recomputation from `places.csv`. (As built, October 16, 2026: `docs/proposals/2026-10-16-world-stage-c1-decisions.md`, WM.E240 to WM.E255, after Jon's ruling WM.60 on the rounding of drawn reals, built first as item 1 of the stage.) -2. **c2a. Pair fluents: the definition, the runtime, and the fixtures.** Pair fluents between entities as given by an event file: symbols and profiles in `definition.json`, `initial_pairs.csv`, pair literals (`pair`, `SOME`, `SOMEOF`, `SAME`) in the literal table, `derive`, pair preconditions, pair effects, and `pair` conditions as written in an event file, compiled literals for explicit entries, `apply` with symmetric writes and the invariant, non-interference over pairs, rows, and columns, the `spatial` block and `spatial_id`, episodes and histories with pair changes, `replay`, the brute-force evaluator over pairs, `tests/fixtures/world/README.md`, and hand-written fixtures 1 to 4, 10, 12 to 14, and 16. Nothing is generated yet: with no event file, a world has no pair fluents. *Accept:* every fixture passes; on a world with pair fluents from an event file, `legal` and `derive` agree with the brute-force evaluator in 1,000 random states, and applying a step's events together equals applying them one at a time in every order, on 1,000 random non-interfering sets; no profile is broken in 1,000 episodes of that world; replay reproduces every pair change; a broken event file fails with an error that names the file and the entry; every world without an event file is byte-identical to stage c1. +2. **c2a. Pair fluents: the definition, the runtime, and the fixtures.** Pair fluents between entities as given by an event file: symbols and profiles in `definition.json`, `initial_pairs.csv`, pair literals (`pair`, `SOME`, `SOMEOF`, `SAME`) in the literal table, `derive`, pair preconditions, pair effects, and `pair` conditions as written in an event file, compiled literals for explicit entries, `apply` with symmetric writes and the invariant, non-interference over pairs, rows, and columns, the `spatial` block and `spatial_id`, episodes and histories with pair changes, `replay`, the brute-force evaluator over pairs, `tests/fixtures/world/README.md`, and hand-written fixtures 1 to 4, 10, 12 to 14, and 16. Nothing is generated yet: with no event file, a world has no pair fluents. *Accept:* every fixture passes; on a world with pair fluents from an event file, `legal` and `derive` agree with the brute-force evaluator in 1,000 random states, and applying a step's events together equals applying them one at a time in every order, on 1,000 random non-interfering sets; no profile is broken in 1,000 episodes of that world; replay reproduces every pair change; a broken event file fails with an error that names the file and the entry; every world without an event file is byte-identical to stage c1. (As built, October 21, 2026: `docs/proposals/2026-10-21-world-stage-c2a-decisions.md`, WM.E256 to WM.E268, after Jon's rulings WM.61 and WM.62 on stage c1's open questions; the rounding of the taxonomy's similarity range was built first as item 1 of the stage.) 3. **c2b. Pair fluents: generation.** Drawn profiles and initial rates, initial pairs, pair literals in preconditions, pair effects, and `pair` conditions at their shares, from their streams; compiled literals in generation; the redraws of contradictions, empty effects, and never-legal event types; derived fluents over `SOME` and `SOMEOF`, with the quota `derived_rules` (review point 3); the `pair_fluents` statistics; generated fixtures; and the tiny and default configurations. *Accept:* on the default and tiny worlds, every pair literal is achievable, no event type has contradicting effects, every add on an asymmetric or functional pair fluent carries its compiled literals, and initial pairs keep every profile; on every seed from 1 to 5 of the default world, at least `derived_rules` derived fluents read a `SOME` or `SOMEOF` literal; `legal` and `derive` agree with the brute-force evaluator in 1,000 random states of the tiny world; no profile is broken in 1,000 statistics episodes; the off settings of "Determinism" give their identities. -4. **c3. Locations.** Location fluents, habitats, the dynamics of place event types (moves and acts, their compiled literals), `SAME` and `SOMEOF` on location fluents, the quota `same_rules`, places in episodes (`scene.extra_places`, `scene.places`, `scene.initial_location`, redraws), histories with places, `uniform_event_type` with place event types, the location statistics, `place_chain_example.yaml`, and hand-written fixtures 5 to 9, 11, and 15. *Accept:* every entity is at exactly one place at every time point of every episode; every move changes the agent's place to the bound place and nothing else of the location fluent; no move goes to the agent's own place; every act happens at the agent's place; with `habitat_strength: 0` the initial places are uniform within a sampling margin stated from the configuration, and with the default, an entity's chance of a place rises with the number of place event types it is able to do there (checked against the weights, within the same kind of margin); the default habitats are not flat (review point 2); on every seed from 1 to 5 of the default world, at least `same_rules` two-place event types require `SAME` on a location fluent (review point 4); the chain example's events occur in the order its preconditions force. The stage records the default world's two-place share, move share, and act share (ruling 4). +4. **c3. Locations.** Location fluents, habitats, the dynamics of place event types (moves and acts, their compiled literals), `SAME` and `SOMEOF` on location fluents, the quota `same_rules`, places in episodes (`scene.extra_places`, `scene.places`, `scene.initial_location`, redraws), histories with places, `uniform_event_type` with place event types, the location statistics, `place_chain_example.yaml`, and hand-written fixtures 5 to 9, 11, and 15. *Accept:* every entity is at exactly one place at every time point of every episode; every move changes the agent's place to the bound place and nothing else of the location fluent; no move goes to the agent's own place; every act happens at the agent's place; with `habitat_strength: 0` the initial places are uniform within a sampling margin stated from the configuration, and with the default, an entity's chance of a place rises with the number of place event types it is able to do there (checked against the weights, within the same kind of margin); the default habitats are not flat (review point 2); on every seed from 1 to 5 of the default world, at least `same_rules` two-place event types require `SAME` on a location fluent (review point 4); the chain example's events occur in the order its preconditions force. The stage records the default world's two-place share, move share, and act share (ruling 4). (Jon's ruling, October 21, 2026, WM.62: in this stage `data/world/tiny.yaml`'s `place_event_types.constraint_families` is changed so that every tiny place requirement reads the place, for example `{patient: 1, cross: 1, key_lock: 1}` with the agent family at 0, because the tests of habitats and of acts at a place need tiny place requirements that read the place; stage c1's seed-1 tiny requirements read the agent alone.) 5. **c4. The corpus at the instance level.** First, the oracle's replay (`tests/corpus/truth_oracle.py`) of pair fluents and places (note 6 of stage b3b). Then the lexicon of places, pair fluents, and `SAME`; prepositional predicates and their word order; places as referents with mentions; pair states and changes (`pair_state`), initial states, results, and blocked events; moves and acts in narratives; comparative descriptions of places; the state, event, and `able_now` sets with the change `object`; and `CORPUS.md`. *Accept:* every pair state, location, and change sentence is true of its scene's history by the oracle; every false item is false; `interpret(tree)` recovers every logical form; with the preposition before and after its noun phrase, trees are correct for all six clause orders; the propositional rendering parses back. 6. **c5a. The corpus at the class level: grammar and statements.** `them`; relative clauses with a prepositional predicate and `grammar.word_order.stranding`; quantified objects of prepositions; causal statements with pair literals, with `SOMEPLACE`, and with `SOME`; no statement for a conditional effect with a `pair` condition (review point 11); profile statements; their JSON forms, renderings, readings, and truth tests. Causal statements appear where documents already call for them (feature documents about event types); profile statements are enumerated and checked here, and enter documents in c5b. *Accept:* every causal and profile statement is true by a re-reading of `definition.json`; the unconditional form of a conditional effect is never stated as true, and no conditional effect with a `pair` condition is stated; with stranding and pied-piping, trees are correct for all six clause orders and `interpret(tree)` recovers every logical form; the propositional rendering parses back. 7. **c5b. Documents, test sets, and the end of the phase.** Documents about pair fluents; places in category and feature documents; capacity facts about places; the causal, profile, and class sets; the regenerated default and tiny runs end to end (world, corpus, word forms, render); the as-built notes in this specification; and the guides. *Accept:* every false causal, profile, and class item is false; every `observed` mark agrees with the corpus's scenes; the role swap never produces a pair for a symmetric pair fluent; the full chain runs on the tiny configuration from the guides' commands alone. diff --git a/python/semantic_world/corpus/world.py b/python/semantic_world/corpus/world.py index 84d846b..c7c2d61 100644 --- a/python/semantic_world/corpus/world.py +++ b/python/semantic_world/corpus/world.py @@ -100,6 +100,13 @@ def load_world(config: Config | WorldSource) -> World: view of it. A run folder is checked against the regenerated world.""" source = config.world if isinstance(config, Config) else config result = define(source.config) + if len(result.pair_fluents): + raise CorpusError( + f"the world {source.config.name} has pair fluents " + f"({', '.join(result.pair_fluents.labels)}), which the corpus cannot state yet: pair " + f"fluents enter the corpus in stage c4 of docs/specs/WORLD_PHASE_C.md. Use a world " + f"whose event file defines no pair fluents." + ) if source.kind == "run": check_run_folder(result, source.path) return World(result, source) diff --git a/python/semantic_world/taxonomy/analysis.py b/python/semantic_world/taxonomy/analysis.py index 7b873d7..a408634 100644 --- a/python/semantic_world/taxonomy/analysis.py +++ b/python/semantic_world/taxonomy/analysis.py @@ -295,6 +295,15 @@ def _duplicates(rows: np.ndarray) -> int: return count +def rounded_similarity(value: float) -> float: + """A similarity of the superordinates as ``summary.yaml`` writes it: rounded to 6 decimals, + as the CSVs are (Jon's ruling WM.61, October 21, 2026: the range comes from a matrix + product, which differs between machines in the 14th digit; the value the bound is checked + against is not rounded), and a rounded zero written as ``0.0``, never ``-0.0``, whose sign + would depend on exactly the noise the rounding removes (Jon's ruling of the same day).""" + return round(float(value), 6) + 0.0 + + def summary_stats( rules: RuleSet, tree: Tree, instances: Instances, warnings: tuple[str, ...] ) -> dict[str, Any]: @@ -314,6 +323,8 @@ def summary_stats( else np.zeros((0, 0), dtype=np.uint8) ) similarity = tree.superordinate_similarity + if similarity is not None: + similarity = (rounded_similarity(similarity[0]), rounded_similarity(similarity[1])) return { "categories": len(tree.categories), "leaves": len(tree.leaves), diff --git a/python/semantic_world/world/__main__.py b/python/semantic_world/world/__main__.py index e721446..ed4f62d 100644 --- a/python/semantic_world/world/__main__.py +++ b/python/semantic_world/world/__main__.py @@ -110,6 +110,8 @@ def _define(args: argparse.Namespace) -> int: places = ( f", {result.places.count} places and {len(result.places.event_types)} place event types" ) + if len(result.pair_fluents): + places += f", {len(result.pair_fluents)} pair fluents" print( f"wrote {folder}: {len(result.taxonomy.instances)} entities, {len(result.fluents)} fluents " f"({len(result.fluents.derived)} derived), {len(result.numeric)} numeric fluents, " diff --git a/python/semantic_world/world/definition.py b/python/semantic_world/world/definition.py index 17edbc4..0fdf380 100644 --- a/python/semantic_world/world/definition.py +++ b/python/semantic_world/world/definition.py @@ -14,7 +14,13 @@ threshold of the patient, a comparison of an entity scalar with a place scalar, or a constraint's output. Place symbols, literals, rules, layers, and event types appear only when the world has places, so with ``places: null`` the definition and its identity are those of the world before -places. +places. Since stage c2a a world with pair fluents (``docs/specs/WORLD_PHASE_C.md``) has symbols +of kind ``pair_fluent`` with their profiles, entity-scope literals of the kinds ``pair_some`` and +``pair_someof`` where a derived fluent's rule reads ``SOME.PAIRFL.k`` or ``SOMEOF.PAIRFL.k`` of +its own entity (computed before the matrices, from the state's pairs), pair literals in the +preconditions and conditions and pair effects in the event-type records, and, when an event file +gives one, a ``spatial`` block with its own ``spatial_id`` outside the rule-set identity. A world +without pair fluents has none of these keys. The rule-set identity is the SHA-256 of the canonical JSON of ``symbols``, ``literals``, ``rules``, and ``event_types``. The agreement test checks every entity rule on every entity and @@ -41,12 +47,25 @@ RoleThreshold, ScalarComparison, ) -from semantic_world.world.dynamics import ROLES, ConditionLiteral, Effect, Literal +from semantic_world.world.dynamics import ( + PAIR_KIND, + PAIR_KINDS, + PAIR_SAME, + PAIR_SOME, + PAIR_SOMEOF, + ROLES, + TWO_ARGUMENT_KINDS, + ConditionLiteral, + Effect, + Literal, + PairLiteral, +) from semantic_world.world.errors import DefinitionError, WorldError from semantic_world.world.event_types import ConstraintSpec, EventTypes from semantic_world.world.fluents import Fluents, ThresholdLiteral from semantic_world.world.identity import ( DEFINITION_VERSION, + canonical_json, read_json, rule_set_id, to_json, @@ -68,6 +87,15 @@ NumericThreshold, empty_numeric, ) +from semantic_world.world.pair_fluents import ( + INITIAL_PAIRS_FILE, + PAIR_FLUENT_KIND, + PairFluent, + PairFluents, + empty_pair_fluents, + initial_pairs_frame, + matrices_from_rows, +) from semantic_world.world.places import ( PLACE_EVENT_TYPE_KIND, PLACE_KIND_OF_TYPE, @@ -134,6 +162,61 @@ def numeric_threshold_literal(label: str, threshold: float) -> LiteralSpec: ) +PAIR_LITERAL_PREFIX = {PAIR_SOME: "SOME.", PAIR_SOMEOF: "SOMEOF."} + + +def pair_literal_spec(kind: str, pair_fluent: str) -> LiteralSpec: + """An entity-scope pair literal (stage c2a), read by a derived fluent's rule of its own + entity: ``SOME.PAIRFL.1`` (the entity's row holds a pair) or ``SOMEOF.PAIRFL.1`` (its + column does).""" + if kind not in PAIR_LITERAL_PREFIX: + raise WorldError(f"a rule reads {kind} literals; a rule reads SOME and SOMEOF only") + key = f"{PAIR_LITERAL_PREFIX[kind]}{pair_fluent}" + return LiteralSpec(key, {"kind": kind, "role": None, "pair_fluent": pair_fluent}) + + +def pair_literal_key_of_rule_input(key: str) -> tuple[str, str] | None: + """``(kind, pair fluent)`` when a rule's input key is a pair literal, ``SOME.PAIRFL.1`` or + ``SOMEOF.PAIRFL.1``; None otherwise.""" + for kind, prefix in PAIR_LITERAL_PREFIX.items(): + if key.startswith(prefix) and key[len(prefix) :].startswith("PAIRFL."): + return kind, key[len(prefix) :] + return None + + +def spatial_identity(block: Mapping[str, Any]) -> str: + """``spatial_id`` (F6): the SHA-256 hash of the canonical JSON of the ``spatial`` block.""" + import hashlib + + return hashlib.sha256(canonical_json(dict(block)).encode("utf-8")).hexdigest() + + +def check_spatial_block(block: Any, pair_fluents: Sequence[str]) -> dict[str, dict[str, Any]]: + """The form of a ``spatial`` block: a mapping from a pair fluent of the definition to a + mapping with a ``detector`` name and a ``parameters`` mapping. The runtime checks nothing + else.""" + if not isinstance(block, Mapping): + raise DefinitionError("the spatial block is not a mapping") + found: dict[str, dict[str, Any]] = {} + for label, entry in block.items(): + if label not in pair_fluents: + raise DefinitionError(f"the spatial block names {label}, which is not a pair fluent") + if ( + not isinstance(entry, Mapping) + or not isinstance(entry.get("detector"), str) + or not isinstance(entry.get("parameters", {}), Mapping) + or set(entry) - {"detector", "parameters"} + ): + raise DefinitionError( + f"the spatial entry of {label} is not {{detector: name, parameters: mapping}}" + ) + found[str(label)] = { + "detector": str(entry["detector"]), + "parameters": dict(entry.get("parameters", {})), + } + return found + + def comparison_literal(item: ScalarComparison) -> LiteralSpec: key = item.key return LiteralSpec( @@ -205,17 +288,27 @@ def _is_place_literal(reads: Mapping[str, Any], place_features: Mapping[str, int _GROUP_ORDER = { - (None, "feature"): 0, - (None, "threshold"): 1, - (None, "fluent"): 2, - (None, "numeric_threshold"): 3, - ("agent", "feature"): 4, - ("agent", "threshold"): 5, - ("patient", "feature"): 6, - ("patient", "threshold"): 7, - ("binding", "comparison"): 8, - ("binding", "constraint"): 9, + key: position + for position, key in enumerate( + ( + (None, "feature"), + (None, "threshold"), + (None, "fluent"), + (None, "numeric_threshold"), + (None, PAIR_SOME), + (None, PAIR_SOMEOF), + ("agent", "feature"), + ("agent", "threshold"), + ("patient", "feature"), + ("patient", "threshold"), + ("binding", "comparison"), + ("binding", "constraint"), + ) + ) } +"""The order of the literal groups in the literal table. The entity-scope pair literals of +stage c2a (``SOME`` and ``SOMEOF`` of the entity) follow the numeric thresholds; a world +without them orders its literals as before.""" def _literal_sort_key( @@ -239,6 +332,8 @@ def _literal_sort_key( return (group, _natural(reads["fluent"]), 0.0) if reads["kind"] == "numeric_threshold": return (group, _natural(reads["fluent"]), reads["threshold"]) + if reads["kind"] in PAIR_KINDS: + return (group, _natural(reads["pair_fluent"]), 0.0) if reads["kind"] == "comparison": return ( group, @@ -278,11 +373,18 @@ class Definition: """The constraints and requirements of the place event types (scope ``place_binding``).""" place_matrices: RuleMatrices | None = None place_binding_matrices: RuleMatrices | None = None + pair_fluents: PairFluents = None # type: ignore[assignment] + """The pair fluents between entities and their initial pairs (stage c2a); none when the + world has none.""" + spatial: dict[str, dict[str, Any]] | None = None + """The ``spatial`` block (F6), from an event file; None when there is none.""" def __post_init__(self) -> None: object.__setattr__(self, "_index", {lit.key: i for i, lit in enumerate(self.literals)}) if self.numeric is None: object.__setattr__(self, "numeric", empty_numeric(len(self.statics.instances))) + if self.pair_fluents is None: + object.__setattr__(self, "pair_fluents", empty_pair_fluents()) if self.places is not None and ( self.place_matrices is None or self.place_binding_matrices is None ): @@ -342,9 +444,14 @@ def event_types_record(self) -> list[dict[str, Any]]: records += [et.record() for et in self.places.event_types] return records + @property + def spatial_id(self) -> str | None: + return None if self.spatial is None else spatial_identity(self.spatial) + def record(self) -> dict[str, Any]: - """The contents of ``definition.json``.""" - return { + """The contents of ``definition.json``. The ``spatial`` block and its ``spatial_id`` + (F6) are written only when the definition has the block; neither is hashed.""" + record = { "version": DEFINITION_VERSION, "rule_set_id": self.rule_set_id, "symbols": list(self.symbols), @@ -353,6 +460,10 @@ def record(self) -> dict[str, Any]: "layers": self.layers_record(), "event_types": self.event_types_record(), } + if self.spatial is not None: + record["spatial"] = {k: dict(v) for k, v in self.spatial.items()} + record["spatial_id"] = self.spatial_id + return record def runtime(self) -> RuntimeDefinition: """The definition as the runtime reads it, built from this definition's own record and @@ -364,7 +475,18 @@ def runtime(self) -> RuntimeDefinition: places = json.loads( to_json(places_frame(self.places).to_dicts(), indent=1, sort_keys=False) ) - return runtime_definition(record, entities, places) + initial_pairs = None + if len(self.pair_fluents): + initial_pairs = json.loads( + to_json( + initial_pairs_frame( + self.pair_fluents, self.taxonomy.instances.labels + ).to_dicts(), + indent=1, + sort_keys=False, + ) + ) + return runtime_definition(record, entities, places, initial_pairs) # Literal values --------------------------------------------------------------------------- @@ -381,17 +503,21 @@ def entity_literal_values( scalars: np.ndarray, base_fluents: np.ndarray, numeric: np.ndarray | None = None, + pairs: Sequence[np.ndarray] | None = None, ) -> np.ndarray: """The base literal matrix of entities: free features from ``values`` (the static feature matrix, by position), thresholds from ``scalars``, base fluents from ``base_fluents`` (``(n, base fluents)``), numeric thresholds from ``numeric`` (``(n, - numeric fluents)``, the initial values when None), and 0 for every literal that a rule - computes or that a binding reads.""" + numeric fluents)``, the initial values when None), the pair literals ``SOME`` and + ``SOMEOF`` from ``pairs`` (one matrix per pair fluent; the initial pairs when None), and + 0 for every literal that a rule computes or that a binding reads.""" n = values.shape[0] matrix = self.entity_matrices.literal_matrix(n) base_index = {label: i for i, label in enumerate(self.fluents.base_labels)} if numeric is None: numeric = self.numeric.initial_values + if pairs is None: + pairs = self.pair_fluents.initial_pairs for i, spec in enumerate(self.literals): reads = spec.reads if reads["role"] is not None or _is_place_literal(reads, self._place_positions()): @@ -408,6 +534,12 @@ def entity_literal_values( elif reads["kind"] == "numeric_threshold": column = numeric[:, self.numeric.index(reads["fluent"])] matrix[:, i] = (column > reads["threshold"]).astype(np.uint8) + elif reads["kind"] == PAIR_SOME: + pair_matrix = pairs[self.pair_fluents.index(reads["pair_fluent"])] + matrix[:, i] = np.asarray(pair_matrix, dtype=bool).any(axis=1).astype(np.uint8) + elif reads["kind"] == PAIR_SOMEOF: + pair_matrix = pairs[self.pair_fluents.index(reads["pair_fluent"])] + matrix[:, i] = np.asarray(pair_matrix, dtype=bool).any(axis=0).astype(np.uint8) return matrix def binding_literal_values( @@ -562,7 +694,11 @@ def _place_symbols(places: Places) -> list[dict[str, Any]]: def _symbols( - statics: StaticWorld, fluents: Fluents, event_types: EventTypes, numeric: NumericFluents + statics: StaticWorld, + fluents: Fluents, + event_types: EventTypes, + numeric: NumericFluents, + pair_fluents: PairFluents | None = None, ) -> list[dict[str, Any]]: taxonomy = statics.taxonomy symbols: list[dict[str, Any]] = [] @@ -599,6 +735,8 @@ def _symbols( ) for numeric_fluent in numeric.fluents: symbols.append(numeric_fluent.symbol_record()) + for pair_fluent in pair_fluents or (): + symbols.append(pair_fluent.symbol_record()) for et in event_types.event_types: symbols.append( { @@ -618,12 +756,20 @@ def build_definition( event_types: EventTypes, numeric: NumericFluents | None = None, places: Places | None = None, + pair_fluents: PairFluents | None = None, + spatial: Mapping[str, Mapping[str, Any]] | None = None, ) -> Definition: """Assemble the definition: the literal table, the rule scopes (entity and binding, and - with places the place and place-binding scopes), the matrices, and the identity.""" + with places the place and place-binding scopes), the matrices, and the identity. With pair + fluents (stage c2a), their symbols, the entity-scope pair literals the derived rules read, + and, when given, the ``spatial`` block.""" taxonomy = statics.taxonomy if numeric is None: numeric = empty_numeric(len(taxonomy.instances)) + if pair_fluents is None: + pair_fluents = empty_pair_fluents() + if spatial is not None: + spatial = check_spatial_block(spatial, pair_fluents.labels) position_of = {f.label: f.position for f in statics.features.features} place_position_of = ( {f.label: f.position for f in places.taxonomy.features.features} @@ -648,6 +794,13 @@ def add(spec: LiteralSpec) -> None: for key in rule.inputs: if key in fluents: add(fluent_literal(key)) + elif (pair_input := pair_literal_key_of_rule_input(key)) is not None: + if pair_input[1] not in pair_fluents: + raise WorldError( + f"the rule for {rule.output} reads {key}, but {pair_input[1]} is not a " + f"pair fluent of the world" + ) + add(pair_literal_spec(*pair_input)) elif key.startswith("NUMFL."): numeric_literal: NumericThreshold = next( t for t in rule.numeric_thresholds if t.key == key @@ -722,7 +875,7 @@ def add(spec: LiteralSpec) -> None: build_matrices(literals, place_binding_rules) if places is not None else None ) symbols = tuple( - _symbols(statics, fluents, event_types, numeric) + _symbols(statics, fluents, event_types, numeric, pair_fluents) + (_place_symbols(places) if places is not None else []) ) @@ -745,6 +898,8 @@ def assemble(identity: str) -> Definition: tuple(place_binding_rules), place_matrices, place_binding_matrices, + pair_fluents, + None if spatial is None else dict(spatial), ) draft = assemble("") @@ -919,11 +1074,17 @@ def entities_frame(definition: Definition) -> pl.DataFrame: def write_definition(definition: Definition, folder: str | Path) -> None: + """Write ``definition.json``, ``entities.csv``, ``places.csv`` in a world with places, and + ``initial_pairs.csv`` in a world with pair fluents (stage c2a).""" folder = Path(folder) write_json(folder / DEFINITION_FILE, definition.record()) entities_frame(definition).write_csv(folder / ENTITIES_FILE) if definition.places is not None: places_frame(definition.places).write_csv(folder / PLACES_FILE) + if len(definition.pair_fluents): + initial_pairs_frame( + definition.pair_fluents, definition.taxonomy.instances.labels + ).write_csv(folder / INITIAL_PAIRS_FILE) # --------------------------------------------------------------------------------------------- @@ -941,9 +1102,19 @@ def write_definition(definition: Definition, folder: str | Path) -> None: *PLACE_STATIC_KINDS, PLACE_SCALAR_KIND, PLACE_EVENT_TYPE_KIND, + PAIR_FLUENT_KIND, ) EVENT_TYPE_KINDS = ("event_type", "event_type_category", PLACE_EVENT_TYPE_KIND) -LITERAL_KINDS = ("feature", "threshold", "fluent", "numeric_threshold", "comparison", "constraint") +LITERAL_KINDS = ( + "feature", + "threshold", + "fluent", + "numeric_threshold", + "comparison", + "constraint", + *PAIR_KINDS, +) +CONDITION_KINDS = ("feature", "fluent", "numeric", *PAIR_KINDS) SCOPES = ("entity", "binding", "place", "place_binding") ENTITY_COLUMNS = ("label", "leaf") @@ -958,13 +1129,20 @@ class EventTypeRecord: kind: str arity: int requirement: str - precondition: tuple[Literal, ...] + precondition: tuple[Literal | PairLiteral, ...] effects: tuple[Effect, ...] @property def roles(self) -> tuple[str, ...]: return ROLES[: self.arity] + @property + def reads_pairs(self) -> bool: + """Whether the precondition or a condition holds a pair literal (stage c2a).""" + return any(lit.pair for lit in self.precondition) or any( + lit.pair for effect in self.effects for lit in effect.condition + ) + @property def is_category(self) -> bool: return self.kind == "event_type_category" @@ -1041,9 +1219,21 @@ class RuntimeDefinition: place_binding_rules: tuple[RuleSpec, ...] = () place_matrices: RuleMatrices | None = None place_binding_matrices: RuleMatrices | None = None + pair_fluents: tuple[PairFluent, ...] = () + """The pair fluents (stage c2a), in symbol order, with their profiles.""" + initial_pairs: tuple[np.ndarray, ...] | None = None + """One Boolean matrix per pair fluent, ``(entities, entities)``: the pairs of + ``initial_pairs.csv``; None when no initial pairs were given (a fixture gives them apart).""" + spatial: dict[str, dict[str, Any]] | None = None + """The ``spatial`` block (F6), or None.""" + spatial_id: str | None = None def __post_init__(self) -> None: object.__setattr__(self, "_entity_index", {v: i for i, v in enumerate(self.entity_labels)}) + object.__setattr__( + self, "_pair_index", {f.label: i for i, f in enumerate(self.pair_fluents)} + ) + object.__setattr__(self, "_pair_cones", _pair_cones(self)) object.__setattr__(self, "_event_types", {et.label: et for et in self.event_types}) object.__setattr__(self, "_base_index", {v: i for i, v in enumerate(self.base_fluents)}) object.__setattr__( @@ -1065,10 +1255,38 @@ def __post_init__(self) -> None: self.initial_numeric, self.place_free_values, self.place_scalar_values, + *(self.initial_pairs or ()), ): if arr is not None: arr.setflags(write=False) + @property + def has_pair_fluents(self) -> bool: + return bool(self.pair_fluents) + + @property + def pair_labels(self) -> tuple[str, ...]: + return tuple(f.label for f in self.pair_fluents) + + def is_pair_fluent(self, label: str) -> bool: + return label in self._pair_index # type: ignore[attr-defined] + + def pair_index(self, label: str) -> int: + try: + return self._pair_index[label] # type: ignore[attr-defined] + except KeyError: + raise WorldError(f"{label} is not a pair fluent of the definition") from None + + def pair_fluent(self, label: str) -> PairFluent: + return self.pair_fluents[self.pair_index(label)] + + def pair_cone(self, fluent: str) -> tuple[tuple[str, str], ...]: + """The rows and columns of its own entity a fluent reads through the pair literals of + its rule, directly or through other derived fluents (stage c2a): ``("row", "PAIRFL.1")`` + for ``SOME.PAIRFL.1`` and ``("column", "PAIRFL.1")`` for ``SOMEOF.PAIRFL.1``; empty for + a base fluent, a numeric fluent, or a rule that reads no pair literal.""" + return self._pair_cones.get(fluent, ()) # type: ignore[attr-defined] + @property def entity_count(self) -> int: return len(self.entity_labels) @@ -1159,6 +1377,35 @@ def cone_of(label: str) -> tuple[str, ...]: return cones +def _pair_cones(definition: RuntimeDefinition) -> dict[str, tuple[tuple[str, str], ...]]: + """Each derived fluent's pair reads (``pair_cone``), through the rules it depends on.""" + literal_of = {spec.key: spec.reads for spec in definition.literals} + rules = {rule.output: rule for rule in definition.entity_rules} + cones: dict[str, tuple[tuple[str, str], ...]] = {} + pair_order = {f.label: i for i, f in enumerate(definition.pair_fluents)} + + def cone_of(label: str) -> tuple[tuple[str, str], ...]: + if label in cones: + return cones[label] + if label not in rules or label not in definition.derived_fluents: + return () + found: set[tuple[str, str]] = set() + for key in rules[label].inputs: + reads = literal_of[key] + if reads["kind"] == PAIR_SOME: + found.add(("row", reads["pair_fluent"])) + elif reads["kind"] == PAIR_SOMEOF: + found.add(("column", reads["pair_fluent"])) + elif reads["kind"] in ("fluent", "numeric_threshold"): + found.update(cone_of(reads["fluent"])) + cones[label] = tuple(sorted(found, key=lambda r: (pair_order[r[1]], r[0]))) + return cones[label] + + for label in definition.derived_fluents: + cone_of(label) + return cones + + def _expect(condition: bool, message: str) -> None: if not condition: raise DefinitionError(message) @@ -1183,21 +1430,35 @@ def runtime_definition( record: Mapping[str, Any], entities: Sequence[Mapping[str, Any]] | pl.DataFrame, places: Sequence[Mapping[str, Any]] | pl.DataFrame | None = None, + initial_pairs: Sequence[Mapping[str, Any]] + | Sequence[Sequence[str]] + | pl.DataFrame + | None = None, ) -> RuntimeDefinition: """Build the runtime's definition from a definition record (the contents of ``definition.json``), an entities table (the rows of ``entities.csv``, or the same rows - without the fluent columns), and, for a world with places, the places table (the rows of - ``places.csv``). Checks the identity and every reference between the tables.""" + without the fluent columns), for a world with places the places table (the rows of + ``places.csv``), and, for a world with pair fluents (stage c2a), the initial pairs (the rows + of ``initial_pairs.csv``, or triples; None when a caller gives the initial pairs apart, as + a fixture does). Checks the identity and every reference between the tables.""" rows = entities.to_dicts() if isinstance(entities, pl.DataFrame) else list(entities) place_rows = places.to_dicts() if isinstance(places, pl.DataFrame) else list(places or []) + pair_rows = ( + initial_pairs.to_dicts() + if isinstance(initial_pairs, pl.DataFrame) + else (None if initial_pairs is None else list(initial_pairs)) + ) try: - return _runtime_definition(record, rows, place_rows) + return _runtime_definition(record, rows, place_rows, pair_rows) except (KeyError, TypeError) as error: raise DefinitionError(f"the definition record is malformed: {error!r}") from None def _runtime_definition( - record: Mapping[str, Any], rows: Sequence[Mapping[str, Any]], place_rows: Sequence + record: Mapping[str, Any], + rows: Sequence[Mapping[str, Any]], + place_rows: Sequence, + pair_rows: Sequence | None = None, ) -> RuntimeDefinition: _expect( record["version"] == DEFINITION_VERSION, @@ -1229,6 +1490,7 @@ def _runtime_definition( place_derived: list[str] = [] place_scalars: list[str] = [] place_event_types: list[str] = [] + pair_fluents: list[PairFluent] = [] for symbol in record["symbols"]: label, kind = str(symbol["label"]), str(symbol["kind"]) _expect(kind in SYMBOL_KINDS, f"the symbol {label} has the unknown kind {kind!r}") @@ -1244,6 +1506,21 @@ def _runtime_definition( place_scalars.append(label) elif kind == PLACE_EVENT_TYPE_KIND: place_event_types.append(label) + elif kind == PAIR_FLUENT_KIND: + _expect( + symbol.get("fluent") is True + and symbol.get("derived") is False + and int(symbol.get("arity", 0)) == 2, + f"the pair fluent {label} is a base fluent of arity 2", + ) + try: + pair_fluents.append(PairFluent.from_record(symbol)) + except (KeyError, TypeError, ValueError, WorldError) as error: + raise DefinitionError(f"the pair fluent {label} is malformed: {error}") from None + _expect( + not pair_fluents[-1].total, + f"the pair fluent {label} is total; a location fluent comes in stage c3", + ) elif kind == "fluent": rate = symbol.get("initial_rate") if symbol["derived"]: @@ -1288,6 +1565,11 @@ def _runtime_definition( not has_places or place_rows is not None, "the definition has places, but no places table was given", ) + pair_labels = [f.label for f in pair_fluents] + _expect( + pair_labels == [f"PAIRFL.{i}" for i in range(1, len(pair_labels) + 1)], + "the pair fluents are not PAIRFL.1 to PAIRFL. in order", + ) # Literals. literals: list[LiteralSpec] = [] @@ -1367,6 +1649,25 @@ def _runtime_definition( float(entry["low"]) if entry["high"] is not None: float(entry["high"]) + elif kind in PAIR_KINDS: + _expect( + entry.get("pair_fluent") in pair_labels, + f"the literal {key} reads {entry.get('pair_fluent')}, which is not a pair fluent", + ) + if kind in TWO_ARGUMENT_KINDS: + _expect( + role == "binding" + and entry.get("first") in ROLES + and entry.get("second") in ROLES + and entry.get("first") != entry.get("second"), + f"the literal {key} is a {kind} literal, which names two roles of a binding", + ) + else: + _expect( + role in (None, *ROLES), + f"the literal {key} has the role {role!r}; a {kind} literal belongs to the " + f"entity scope (role null) or names one role", + ) else: _expect( role == "binding", f"the literal {key} reads a constraint with the role {role!r}" @@ -1474,8 +1775,11 @@ def _runtime_definition( requirement in constraint_outputs, f"the requirement {requirement} of {label} has no binding rule", ) - precondition = [] + precondition: list[Literal | PairLiteral] = [] for lit in entry["precondition"]["literals"]: + if lit.get("kind") in PAIR_KINDS: + precondition.append(_pair_literal(label, lit, arity, pair_fluents, "precondition")) + continue _expect( lit["role"] in ROLES[:arity], f"the precondition of {label} reads the role {lit['role']!r}", @@ -1507,11 +1811,32 @@ def _runtime_definition( precondition.append(Literal(str(lit["role"]), str(lit["fluent"]), bool(lit["value"]))) effects = [] for effect in entry["effects"]: + second: str | None = None + if effect.get("kind") == PAIR_KIND: + _expect( + effect.get("pair_fluent") in pair_labels, + f"an effect of {label} writes {effect.get('pair_fluent')}, which is not a " + f"pair fluent", + ) + _expect( + effect.get("first") in ROLES[:arity] + and effect.get("second") in ROLES[:arity] + and effect["first"] != effect["second"], + f"the pair effect of {label} on {effect.get('pair_fluent')} names the roles " + f"{effect.get('first')!r} and {effect.get('second')!r}", + ) + role_of_effect, fluent_of_effect = str(effect["first"]), str(effect["pair_fluent"]) + second = str(effect["second"]) + value: bool | float = bool(effect["value"]) + else: + role_of_effect, fluent_of_effect = str(effect["role"]), str(effect["fluent"]) _expect( - effect["role"] in ROLES[:arity], - f"an effect of {label} writes the role {effect['role']!r}", + role_of_effect in ROLES[:arity], + f"an effect of {label} writes the role {role_of_effect!r}", ) - if "amount" in effect: + if second is not None: + pass + elif "amount" in effect: _expect( effect["fluent"] in numeric_fluents, f"an effect of {label} adds to {effect['fluent']}, which is not a numeric " @@ -1526,15 +1851,18 @@ def _runtime_definition( _expect( amount > 0, f"a numeric effect of {label} has a constant that is not above 0" ) - value: bool | float = amount if effect["operator"] == "+=" else -amount + value = amount if effect["operator"] == "+=" else -amount else: _expect( effect["fluent"] in base_fluents, f"an effect of {label} writes {effect['fluent']}, which is not a base fluent", ) value = bool(effect["value"]) - condition = [] + condition: list[ConditionLiteral | PairLiteral] = [] for lit in (effect.get("condition") or {}).get("literals", ()): + if lit.get("kind") in PAIR_KINDS: + condition.append(_pair_literal(label, lit, arity, pair_fluents, "condition")) + continue key = f"{lit.get('role')}.{lit.get('symbol')}" _expect( lit["role"] in ROLES[:arity], @@ -1567,11 +1895,12 @@ def _runtime_definition( ConditionLiteral(str(lit["role"]), str(lit["symbol"]), bool(lit["value"])) ) _expect( - len({(c.role, c.symbol) for c in condition}) == len(condition), + len({(c.role, c.symbol) for c in condition if not c.pair}) + == sum(1 for c in condition if not c.pair), f"the condition of an effect of {label} reads a symbol twice", ) effects.append( - Effect(str(effect["role"]), str(effect["fluent"]), value, tuple(condition)) + Effect(role_of_effect, fluent_of_effect, value, tuple(condition), second) ) event_types.append( EventTypeRecord( @@ -1672,6 +2001,41 @@ def _runtime_definition( else: _expect(not place_rows, "a places table was given, but the definition has no places") + # The spatial block (F6). + spatial = None + spatial_id = None + if "spatial" in record or "spatial_id" in record: + _expect( + "spatial" in record and "spatial_id" in record, + "a definition with a spatial block carries both spatial and spatial_id", + ) + spatial = check_spatial_block(record["spatial"], pair_labels) + spatial_id = spatial_identity(spatial) + _expect( + record["spatial_id"] == spatial_id, + f"the definition says its spatial block hashes to {record['spatial_id']}, but it " + f"hashes to {spatial_id}", + ) + + # The initial pairs (stage c2a). + initial_pairs = None + if pair_rows is not None: + _expect( + pair_fluents or not pair_rows, + "initial pairs were given, but the definition has no pair fluents", + ) + entity_index = {label: i for i, label in enumerate(labels)} + try: + initial_pairs = matrices_from_rows(pair_rows, pair_fluents, entity_index) + except WorldError as error: + raise DefinitionError(str(error)) from None + from semantic_world.world.pair_fluents import profile_violations + + for fluent, matrix in zip(pair_fluents, initial_pairs, strict=True): + violations = profile_violations(fluent, matrix) + if violations: + raise DefinitionError(f"the initial pairs break a profile: {violations[0]}") + definition = RuntimeDefinition( record=dict(record), rule_set_id=identity, @@ -1708,10 +2072,39 @@ def _runtime_definition( place_binding_rules=tuple(place_binding_rules), place_matrices=matrices["place"], place_binding_matrices=matrices["place_binding"], + pair_fluents=tuple(pair_fluents), + initial_pairs=initial_pairs, + spatial=spatial, + spatial_id=spatial_id, ) return definition +def _pair_literal( + label: str, lit: Mapping[str, Any], arity: int, pair_fluents: Sequence[PairFluent], where: str +) -> PairLiteral: + """A pair literal of an event-type record, checked: a pair fluent of the definition, roles + of the event type, and ``SAME`` on a functional pair fluent.""" + try: + literal = PairLiteral.from_record(lit) + except (KeyError, TypeError, ValueError) as error: + raise DefinitionError( + f"a pair literal of the {where} of {label} is malformed: {error}" + ) from None + by_label = {f.label: f for f in pair_fluents} + _expect( + literal.pair_fluent in by_label, + f"the {where} of {label} reads {literal.pair_fluent}, which is not a pair fluent", + ) + for role in literal.roles: + _expect(role in ROLES[:arity], f"the {where} of {label} reads the role {role!r}") + _expect( + literal.kind != PAIR_SAME or by_label[literal.pair_fluent].functional, + f"the {where} of {label} reads {literal.key}, but {literal.pair_fluent} is not functional", + ) + return literal + + def _bit_matrix(rows: Sequence[Mapping[str, Any]], columns: Sequence[str]) -> np.ndarray: matrix = np.zeros((len(rows), len(columns)), dtype=np.uint8) for i, row in enumerate(rows): @@ -1748,7 +2141,15 @@ def load_definition(folder: str | Path) -> RuntimeDefinition: if not places_path.is_file(): raise DefinitionError(f"{places_path} is missing: the definition has places") places = pl.read_csv(places_path, schema_overrides={"label": pl.Utf8, "leaf": pl.Utf8}) - return runtime_definition(record, entities, places) + initial_pairs = None + if any(s.get("kind") == PAIR_FLUENT_KIND for s in record.get("symbols", ())): + pairs_path = folder / INITIAL_PAIRS_FILE + if not pairs_path.is_file(): + raise DefinitionError(f"{pairs_path} is missing: the definition has pair fluents") + initial_pairs = pl.read_csv( + pairs_path, schema_overrides={c: pl.Utf8 for c in ("pair_fluent", "first", "second")} + ) + return runtime_definition(record, entities, places, initial_pairs) __all__ = [ @@ -1760,7 +2161,9 @@ def load_definition(folder: str | Path) -> RuntimeDefinition: "RuntimeDefinition", "build_definition", "check_definition", + "check_spatial_block", "constraint_literals", + "spatial_identity", "entities_frame", "load_definition", "runtime_definition", diff --git a/python/semantic_world/world/dynamics.py b/python/semantic_world/world/dynamics.py index 2bf75d3..25edd03 100644 --- a/python/semantic_world/world/dynamics.py +++ b/python/semantic_world/world/dynamics.py @@ -15,11 +15,20 @@ A numeric threshold is written in a text with ``repr``, the shortest form that reads back to the same float64; the records of ``definition.json`` write every float with 17 significant digits. + +Since stage c2a (``docs/specs/WORLD_PHASE_C.md``, "Literals over pairs") a precondition or a +condition may hold a pair literal, :class:`PairLiteral`: ``PAIRFL.1(agent, patient)`` (the pair +holds), ``SOME.PAIRFL.1(agent)`` (the row of the role holds a pair), ``SOMEOF.PAIRFL.1(patient)`` +(the column does), or ``SAME.PAIRFL.3(agent, patient)`` (the two roles share a partner, on a +functional pair fluent), each positive or negated; a compiled literal (F4) carries its origin, +``profile``. An effect may add or delete a pair between the roles: ``PAIRFL.1(agent, patient) +:= 1`` or ``:= 0`` (an :class:`Effect` whose ``second`` names the second role). """ from __future__ import annotations import re +from collections.abc import Mapping from dataclasses import dataclass from typing import Any @@ -35,12 +44,31 @@ r"^\s*(agent|patient)\.(NUMFL\.\d+)\s*(\+=|-=)\s*((?:\d+\.?\d*|\.\d+)(?:[eE][-+]?\d+)?)\s*$" ) _CONDITIONAL = re.compile(r"^\s*when\s+(.+?)\s+then\s+(.+)$", re.DOTALL) +_PAIR_LITERAL = re.compile( + r"^\s*(?:(SOME|SOMEOF|SAME)\.)?(PAIRFL\.\d+)\s*\(\s*(agent|patient)\s*" + r"(?:,\s*(agent|patient)\s*)?\)\s*$" +) +_PAIR_EFFECT = re.compile( + r"^\s*(PAIRFL\.\d+)\s*\(\s*(agent|patient)\s*,\s*(agent|patient)\s*\)\s*:=\s*([01])\s*$" +) FEATURE_KIND = "feature" FLUENT_KIND = "fluent" NUMERIC_KIND = "numeric" NUMERIC_PREFIX = "NUMFL." +PAIR_PREFIX = "PAIRFL." ABOVE = ">" AT_OR_BELOW = "<=" +PAIR_KIND = "pair" +PAIR_SOME = "pair_some" +PAIR_SOMEOF = "pair_someof" +PAIR_SAME = "pair_same" +PAIR_KINDS = (PAIR_KIND, PAIR_SOME, PAIR_SOMEOF, PAIR_SAME) +"""The kinds of pair literal (stage c2a), as the literal table, the precondition records, and +the condition records name them.""" +PAIR_KIND_OF_PREFIX = {None: PAIR_KIND, "SOME": PAIR_SOME, "SOMEOF": PAIR_SOMEOF, "SAME": PAIR_SAME} +PAIR_PREFIX_OF_KIND = {kind: prefix for prefix, kind in PAIR_KIND_OF_PREFIX.items()} +TWO_ARGUMENT_KINDS = (PAIR_KIND, PAIR_SAME) +PROFILE_ORIGIN = "profile" def _operator(above: bool) -> str: @@ -62,6 +90,14 @@ class Literal: def numeric(self) -> bool: return self.threshold is not None + @property + def pair(self) -> bool: + return False + + @property + def compiled(self) -> bool: + return False + @property def operator(self) -> str: return _operator(self.value) @@ -134,6 +170,10 @@ def kind(self) -> str: def numeric(self) -> bool: return self.kind == NUMERIC_KIND + @property + def pair(self) -> bool: + return False + @property def operator(self) -> str: return _operator(self.value) @@ -191,32 +231,220 @@ def parse(cls, text: str) -> ConditionLiteral: return cls.parse_key(part, value) -def condition_text(literals: tuple[ConditionLiteral, ...]) -> str: +@dataclass(frozen=True, order=True) +class PairLiteral: + """One pair literal (stage c2a) of a precondition or a condition: ``kind`` is ``pair``, + ``pair_some``, ``pair_someof``, or ``pair_same``; ``first`` is the role of the first argument + (the only one of ``SOME`` and ``SOMEOF``) and ``second`` the role of the second argument, or + None; ``value`` is True for the positive literal and False for its negation; ``origin`` is + None for a literal the file or the generator gave, and ``profile`` for a compiled literal + (F4). ``role`` and ``fluent`` name the first role and the pair fluent, so that code that + counts literals by role reads a pair literal as the other kinds.""" + + kind: str + pair_fluent: str + first: str + second: str | None + value: bool + origin: str | None = None + + def __post_init__(self) -> None: + if self.kind not in PAIR_KINDS: + raise ValueError(f"unknown pair literal kind {self.kind!r}") + two = self.kind in TWO_ARGUMENT_KINDS + if two and self.second is None: + raise ValueError(f"a {self.kind} literal names two roles") + if not two and self.second is not None: + raise ValueError(f"a {self.kind} literal names one role") + if two and self.first == self.second: + raise ValueError(f"a {self.kind} literal names two different roles") + + @property + def role(self) -> str: + return self.first + + @property + def fluent(self) -> str: + return self.pair_fluent + + @property + def symbol(self) -> str: + return self.pair_fluent + + @property + def numeric(self) -> bool: + return False + + @property + def pair(self) -> bool: + return True + + @property + def compiled(self) -> bool: + return self.origin is not None + + @property + def roles(self) -> tuple[str, ...]: + return (self.first,) if self.second is None else (self.first, self.second) + + @property + def positive(self) -> str: + """The literal's text without a negation: ``SAME.PAIRFL.3(agent, patient)``.""" + prefix = PAIR_PREFIX_OF_KIND[self.kind] + head = f"{prefix}.{self.pair_fluent}" if prefix else self.pair_fluent + return f"{head}({', '.join(self.roles)})" + + @property + def key(self) -> str: + return self.positive + + @property + def text(self) -> str: + return self.positive if self.value else f"NOT {self.positive}" + + def negated(self) -> PairLiteral: + return PairLiteral(self.kind, self.pair_fluent, self.first, self.second, not self.value) + + def plain(self) -> PairLiteral: + """The same literal without its origin.""" + return PairLiteral(self.kind, self.pair_fluent, self.first, self.second, self.value) + + def record(self) -> dict[str, Any]: + entry: dict[str, Any] = {"kind": self.kind, "pair_fluent": self.pair_fluent} + if self.second is None: + entry["role"] = self.first + else: + entry["first"] = self.first + entry["second"] = self.second + entry["value"] = bool(self.value) + if self.origin is not None: + entry["origin"] = self.origin + return entry + + @classmethod + def from_record(cls, data: Mapping[str, Any]) -> PairLiteral: + kind = str(data["kind"]) + if "role" in data and "first" not in data: + first, second = str(data["role"]), None + else: + first, second = str(data["first"]), str(data["second"]) + origin = data.get("origin") + return cls( + kind, + str(data["pair_fluent"]), + first, + second, + bool(data["value"]), + None if origin is None else str(origin), + ) + + @classmethod + def match(cls, text: str, value: bool = True) -> PairLiteral | None: + """The pair literal a text names, ``SOME.PAIRFL.1(agent)`` or ``PAIRFL.2(patient, + agent)``, with the given value, or None when the text is not one.""" + match = _PAIR_LITERAL.match(text) + if match is None: + return None + kind = PAIR_KIND_OF_PREFIX[match.group(1)] + first, second = match.group(3), match.group(4) + two = kind in TWO_ARGUMENT_KINDS + if two and second is None: + raise ValueError(f"{text.strip()!r}: a {kind} literal names two roles") + if not two and second is not None: + raise ValueError(f"{text.strip()!r}: a {kind} literal names one role") + if two and first == second: + raise ValueError(f"{text.strip()!r}: a {kind} literal names two different roles") + return cls(kind, match.group(2), first, second, value) + + @classmethod + def parse(cls, text: str) -> PairLiteral: + """A pair literal with an optional ``NOT``.""" + part = text.strip() + value = True + if part.upper().startswith("NOT "): + value = False + part = part[4:].strip() + literal = cls.match(part, value) + if literal is None: + raise ValueError( + f"{text!r} is not a pair literal (PAIRFL.(agent, patient), " + f"SOME.PAIRFL.(agent), SOMEOF.PAIRFL.(patient), or " + f"SAME.PAIRFL.(agent, patient), with or without NOT)" + ) + return literal + + +AnyLiteral = Literal | PairLiteral +"""A precondition literal: a Boolean or numeric literal, or, since stage c2a, a pair literal.""" +AnyConditionLiteral = ConditionLiteral | PairLiteral + + +def is_pair_literal(literal: Any) -> bool: + return isinstance(literal, PairLiteral) + + +def condition_text(literals: tuple[AnyConditionLiteral, ...]) -> str: return " AND ".join(lit.text for lit in literals) -def parse_condition(text: str) -> tuple[ConditionLiteral, ...]: +def parse_condition(text: str) -> tuple[AnyConditionLiteral, ...]: """A conjunction of condition literals, ``agent.PROPERTY.4 AND NOT patient.BOOLFL.2 AND - agent.NUMFL.1 > 0.75``.""" - return tuple(ConditionLiteral.parse(part) for part in text.split(" AND ")) + agent.NUMFL.1 > 0.75``, or, since stage c2a, pair literals (``PAIRFL.1(agent, patient)``, + ``NOT SOME.PAIRFL.2(agent)``).""" + literals: list[AnyConditionLiteral] = [] + for part in text.split(" AND "): + stripped = part.strip() + bare = stripped[4:].strip() if stripped.upper().startswith("NOT ") else stripped + if _PAIR_LITERAL.match(bare): + literals.append(PairLiteral.parse(stripped)) + else: + literals.append(ConditionLiteral.parse(part)) + return tuple(literals) @dataclass(frozen=True, order=True) class Effect: """One effect: a base Boolean fluent of one role is set to a value, or (a numeric effect, on a ``NUMFL`` fluent) a constant is added to a numeric fluent of one role or subtracted - from it, ``value`` being the signed constant; under a condition when the effect has one (an - empty tuple is no condition: the effect always applies).""" + from it, ``value`` being the signed constant; or, since stage c2a (a pair effect, on a + ``PAIRFL`` fluent, with ``second`` the role of the second argument), the pair from the role + to the second role is added (``value`` True) or deleted (False); under a condition when the + effect has one (an empty tuple is no condition: the effect always applies).""" role: str fluent: str value: bool | float - condition: tuple[ConditionLiteral, ...] = () + condition: tuple[AnyConditionLiteral, ...] = () + second: str | None = None + + def __post_init__(self) -> None: + if self.pair and self.second is None: + raise ValueError(f"a pair effect on {self.fluent} names a second role") + if self.pair and self.second == self.role: + raise ValueError(f"a pair effect on {self.fluent} names two different roles") + if not self.pair and self.second is not None: + raise ValueError(f"an effect on {self.fluent} names one role") @property def numeric(self) -> bool: return self.fluent.startswith(NUMERIC_PREFIX) + @property + def pair(self) -> bool: + """A pair effect (stage c2a): an add or a delete of a pair between the roles.""" + return self.fluent.startswith(PAIR_PREFIX) + + @property + def roles(self) -> tuple[str, ...]: + return (self.role,) if self.second is None else (self.role, self.second) + + @property + def literal(self) -> PairLiteral: + """The pair literal a pair effect makes true: ``PAIRFL.1(agent, patient)`` for an add, + its negation for a delete.""" + assert self.second is not None + return PairLiteral(PAIR_KIND, self.fluent, self.role, self.second, bool(self.value)) + @property def amount(self) -> float: """A numeric effect's signed constant.""" @@ -228,6 +456,8 @@ def operator(self) -> str: @property def key(self) -> str: + if self.pair: + return f"{self.fluent}({self.role}, {self.second})" return f"{self.role}.{self.fluent}" @property @@ -237,7 +467,7 @@ def conditional(self) -> bool: @property def unconditional(self) -> Effect: """The same effect without its condition.""" - return Effect(self.role, self.fluent, self.value) + return Effect(self.role, self.fluent, self.value, (), self.second) @property def assignment(self) -> str: @@ -257,7 +487,16 @@ def record(self) -> dict[str, Any]: conditions is unchanged; a numeric effect writes its ``operator`` and its ``amount`` in place of a ``value``.""" entry: dict[str, Any] - if self.numeric: + if self.pair: + entry = { + "kind": PAIR_KIND, + "pair_fluent": self.fluent, + "first": self.role, + "second": self.second, + "value": bool(self.value), + "expression": self.text, + } + elif self.numeric: entry = { "role": self.role, "fluent": self.fluent, @@ -291,6 +530,11 @@ def parse(cls, text: str) -> Effect: plain = _EFFECT.match(text) if plain is not None: return cls(plain.group(1), plain.group(2), plain.group(3) == "1", condition) + pair = _PAIR_EFFECT.match(text) + if pair is not None: + if pair.group(2) == pair.group(3): + raise ValueError(f"{text!r}: a pair effect names two different roles") + return cls(pair.group(2), pair.group(1), pair.group(4) == "1", condition, pair.group(3)) numeric = _NUMERIC_EFFECT.match(text) if numeric is not None: amount = float(numeric.group(4)) @@ -300,9 +544,10 @@ def parse(cls, text: str) -> Effect: return cls(numeric.group(1), numeric.group(2), signed, condition) raise ValueError( f"{text!r} is not an effect (write agent.BOOLFL. := 0 or := 1, agent.NUMFL. " - f"+= 0.3 or -= 0.3, or when then )" + f"+= 0.3 or -= 0.3, PAIRFL.(agent, patient) := 0 or := 1, or when " + f"then )" ) -def precondition_text(literals: tuple[Literal, ...]) -> str: +def precondition_text(literals: tuple[AnyLiteral, ...]) -> str: return " AND ".join(lit.text for lit in literals) if literals else "TRUE" diff --git a/python/semantic_world/world/episodes.py b/python/semantic_world/world/episodes.py index 5255773..407665c 100644 --- a/python/semantic_world/world/episodes.py +++ b/python/semantic_world/world/episodes.py @@ -5,9 +5,12 @@ ``scene.size`` others without replacement, each weighted by the thematic relatedness and the taxonomic similarity of its leaf and the seed's leaf, and a constant. It starts from the participants' initial values in ``entities.csv``, or redraws them at each fluent's initial rate -(``scene.initial: redraw``). It runs a drawn number of steps. At each step a Poisson number of -events is drawn one at a time by the selection policy among the events that are legal in the -state and do not interfere with the events already chosen; then the runtime applies the step. +(``scene.initial: redraw``); in a world with pair fluents (stage c2a) it keeps the initial pairs +among its participants and no other pair (WM.38), or redraws them at each pair fluent's rate +after every other draw of its part. It runs a drawn number of steps. At each step a Poisson +number of events is drawn one at a time by the selection policy among the events that are legal +in the state and do not interfere with the events already chosen; then the runtime applies the +step. When no event is legal at a time point, the episode ends there, quiescent. Every episode draws from its own part of a stream, named by its label, so it depends only on the @@ -36,14 +39,17 @@ from semantic_world.world.history import ( DRIFT, EPISODES_FILE, + AnyChange, Change, History, HistoryEvent, Step, changes_of, numeric_changes_of, + pair_changes_of, write_histories, ) +from semantic_world.world.pair_fluents import draw_initial_pairs, pair_rows from semantic_world.world.policies import DEFAULT_POLICY, StepContext, policy from semantic_world.world.runtime import ( Event, @@ -350,9 +356,11 @@ def __init__( if definition.initial_values is not None: self.start = initial_state(definition) elif self.settings.initial == "redraw": + n = definition.entity_count self.start = State( - np.zeros((definition.entity_count, len(definition.base_fluents)), dtype=np.uint8), - np.zeros((definition.entity_count, len(definition.numeric_fluents))), + np.zeros((n, len(definition.base_fluents)), dtype=np.uint8), + np.zeros((n, len(definition.numeric_fluents))), + tuple(np.zeros((n, n), dtype=bool) for _ in definition.pair_fluents), ) else: raise WorldError( @@ -394,10 +402,15 @@ def draw_participants(self, rng: np.random.Generator, seed: int) -> tuple[int, . def _initial(self, rng: np.random.Generator, participants: tuple[int, ...]) -> State: """The participants' initial values: the definition's, or redrawn (``scene.initial: redraw``), the Boolean fluents at their initial rates and then the numeric fluents from - their initial distributions, participant by participant.""" - if self.settings.initial == "keep": - return self.start + their initial distributions, participant by participant. The pairs (stage c2a): the + initial pairs among the participants, every other pair dropped (an episode keeps the + pairs among its participants, WM.38); under ``redraw``, drawn again among the + participants (in entity order) at each pair fluent's rate by the procedure of the initial + pairs, after every other draw of the episode's part.""" definition = self.definition + pairs = self._participant_pairs(participants) + if self.settings.initial == "keep": + return State(self.start.values, self.start.numeric, pairs) values = np.array(self.start.values, dtype=np.uint8, copy=True) numeric = np.array(self.start.numeric, dtype=np.float64, copy=True) for entity in participants: @@ -407,7 +420,25 @@ def _initial(self, rng: np.random.Generator, participants: tuple[int, ...]) -> S for j, fluent in enumerate(definition.numeric_fluents): low, high = definition.numeric_ranges[fluent] numeric[entity, j] = definition.numeric_initial[fluent].draw(rng, low, high, 1)[0] - return State(values, numeric) + if definition.pair_fluents: + rows = np.array(sorted(participants), dtype=np.intp) + redrawn = [] + for fluent in definition.pair_fluents: + local = draw_initial_pairs(fluent, rows.size, rng) + matrix = np.zeros((definition.entity_count, definition.entity_count), dtype=bool) + matrix[np.ix_(rows, rows)] = local + redrawn.append(matrix) + pairs = tuple(redrawn) + return State(values, numeric, pairs) + + def _participant_pairs(self, participants: tuple[int, ...]) -> tuple[np.ndarray, ...]: + """The definition's initial pairs restricted to the participants.""" + if not self.definition.pair_fluents: + return () + mask = np.zeros(self.definition.entity_count, dtype=bool) + mask[list(participants)] = True + keep = np.outer(mask, mask) + return tuple(matrix & keep for matrix in self.start.pairs) def _legal(self, state: State, participants: tuple[int, ...]) -> dict[str, list]: return { @@ -440,6 +471,11 @@ def episode(self, rng: np.random.Generator, label: str, seed: int | None = None) if definition.numeric_fluents else {} ) + initial_pairs = ( + tuple(pair_rows(definition.pair_fluents, state.pairs, labels)) + if definition.pair_fluents + else None + ) recorded: list[Step] = [] quiescent = False count = 0 @@ -477,6 +513,7 @@ def episode(self, rng: np.random.Generator, label: str, seed: int | None = None) + _event_numeric_changes( definition, state, result.after_effects, event, event_label ) + + pair_changes_of(definition, state, after, event) ), ) ) @@ -503,6 +540,7 @@ def episode(self, rng: np.random.Generator, label: str, seed: int | None = None) f"TIME.{len(recorded) + 1}", quiescent, initial_numeric, + initial_pairs, ) def run( @@ -526,13 +564,13 @@ def run( def _event_changes( definition: RuntimeDefinition, before: State, after: State, event: Event -) -> list[Change]: +) -> list[AnyChange]: """The base Boolean fluents the event changed: of the fluents it writes, those whose value differs between the states (non-interfering events write distinct fluents, so each change has one event).""" et = definition.event_type(event.event_type) - entities = sorted({event.binding[et.roles.index(e.role)] for e in et.effects}) - written = {(event.binding[et.roles.index(e.role)], e.fluent) for e in et.effects} + entities = sorted({event.binding[et.roles.index(e.role)] for e in et.effects if not e.pair}) + written = {(event.binding[et.roles.index(e.role)], e.fluent) for e in et.effects if not e.pair} return [ change for change in changes_of(definition, before, after, entities) diff --git a/python/semantic_world/world/errors.py b/python/semantic_world/world/errors.py index 5e361fe..75a5880 100644 --- a/python/semantic_world/world/errors.py +++ b/python/semantic_world/world/errors.py @@ -23,7 +23,8 @@ class DefinitionError(WorldError): class StepError(WorldError): """``apply`` refuses a step. ``kind`` names the kind of error as the conformance fixtures - write it: ``illegal`` or ``interference``.""" + write it: ``illegal`` or ``interference``; ``profile`` (stage c2a) never occurs with a + correct definition, so no fixture ends in it.""" kind = "" @@ -41,3 +42,12 @@ class InterferenceError(StepError): fluent the precondition reads), or the same event occurs twice.""" kind = "interference" + + +class ProfileViolationError(StepError): + """After a step, a pair fluent's profile does not hold (stage c2a): a symmetric pair without + its reverse, an asymmetric pair with its reverse, a functional row with two pairs, or a pair + of an entity with itself. The compiled literals (F4) make this impossible for a correct + definition, so the error marks a broken definition, never a broken step.""" + + kind = "profile" diff --git a/python/semantic_world/world/event_file.py b/python/semantic_world/world/event_file.py index 6cda660..d4c183a 100644 --- a/python/semantic_world/world/event_file.py +++ b/python/semantic_world/world/event_file.py @@ -5,11 +5,19 @@ precondition, and effects by hand. Event types without an entry are sampled. Checks that need the generated world (does the event type exist, is a fluent derived, is a literal achievable) happen when the file is applied, in ``event_types.py``; every error names the file and the entry. + +Since stage c2a (``docs/specs/WORLD_PHASE_C.md``, "Explicit event files") an event file may +define pair fluents between entities (``pair_fluents: {PAIRFL.1: {symmetry, functional, +initial_rate}}``), read pair literals in preconditions and conditions, add or delete pairs in +effects, and give the ``spatial`` block (a detector and its parameters for a pair fluent). A +location fluent (``total: true``), a ``place_event_types`` entry, and a ``locations`` key come in +stage c3 and are errors here. """ from __future__ import annotations import re +from collections.abc import Mapping from dataclasses import dataclass, field from pathlib import Path from typing import Any @@ -27,11 +35,18 @@ Var, parse_expression, ) -from semantic_world.world.dynamics import Effect, Literal +from semantic_world.world.dynamics import AnyLiteral, Effect, Literal, PairLiteral +from semantic_world.world.pair_fluents import SYMMETRIES _EVENT_TYPE = re.compile(r"^EVENTTYPE1\.\d+$|^EVENTTYPE2\.\d+(\.\d+)*$") _FLUENT = re.compile(r"^BOOLFL\.\d+$") _NUMERIC = re.compile(r"^NUMFL\.\d+$") +_PAIR = re.compile(r"^PAIRFL\.\d+$") +_PAIR_TEXT = re.compile( + r"(?:(?:SOME|SOMEOF|SAME)\.)?PAIRFL\.\d+\s*\(\s*(?:agent|patient)\s*" + r"(?:,\s*(?:agent|patient)\s*)?\)" +) +_PLACEHOLDER = "PAIRLITERAL_" @dataclass(frozen=True) @@ -40,8 +55,9 @@ class ExplicitEventType: requirement: Expr | None """A requirement over static literals of the roles (``agent.PROPERTY.3``), or None.""" requirement_text: str | None - precondition: tuple[Literal, ...] | None - """None when the file gives no precondition (it is sampled); an empty tuple is TRUE.""" + precondition: tuple[AnyLiteral, ...] | None + """None when the file gives no precondition (it is sampled); an empty tuple is TRUE. A + literal may be a pair literal (stage c2a).""" effects: tuple[Effect, ...] | None field: str @@ -55,6 +71,12 @@ class EventFile: numeric: dict[str, dict[str, Any]] = field(default_factory=dict) """Named numeric fluents (stage b2): each with an ``initial`` distribution and a ``drift``, either optional.""" + pair_fluents: dict[str, dict[str, Any]] = field(default_factory=dict) + """The pair fluents between entities (stage c2a): each ``{symmetry, functional, + initial_rate}``, in label order.""" + spatial: dict[str, dict[str, Any]] | None = None + """The ``spatial`` block (F6): for a pair fluent, ``{detector, parameters}``; None when the + file gives none.""" def load_event_file(path: str | Path) -> EventFile: @@ -74,8 +96,17 @@ def event_file_from_mapping(data: Any, *, source: str = "") -> EventFil if data is None: data = {} root = _Node(source, "", data) + for key in ("place_event_types", "locations"): + if key in root.data: + raise root.error( + key, + f"{key} in an event file comes in stage c3; stage c2a defines pair fluents " + f"between entities only", + ) fluents: dict[str, float] = {} numeric: dict[str, dict[str, Any]] = {} + pair_fluents = _read_pair_fluents(root) + spatial = _read_spatial(root, pair_fluents) fluents_node = root.mapping("fluents") for label in list(fluents_node.data): if isinstance(label, str) and _NUMERIC.match(label): @@ -108,7 +139,67 @@ def event_file_from_mapping(data: Any, *, source: str = "") -> EventFil entry.finish() types_node.finish() root.finish() - return EventFile(fluents, event_types, source, numeric) + return EventFile(fluents, event_types, source, numeric, pair_fluents, spatial) + + +def _read_pair_fluents(root: _Node) -> dict[str, dict[str, Any]]: + """The ``pair_fluents`` block: ``PAIRFL.: {symmetry: symmetric | asymmetric | general, + functional: bool, initial_rate}``, in label order. ``total: true`` (a location fluent) is an + error until stage c3.""" + node = root.mapping("pair_fluents") + found: dict[str, dict[str, Any]] = {} + for label in list(node.data): + if not isinstance(label, str) or not _PAIR.match(label): + raise node.error(label, "expected a pair fluent label PAIRFL.") + entry = node.mapping(label) + if "total" in entry.data: + if entry.get("total") is not False: + raise entry.error( + "total", + "a total pair fluent is a location fluent, which comes in stage c3; a pair " + "fluent between entities is never total", + ) + entry.get("total") + found[label] = { + "symmetry": entry.choice("symmetry", _MISSING, SYMMETRIES), + "functional": entry.bool("functional", _MISSING), + "initial_rate": float(entry.probability("initial_rate", _MISSING)), + } + entry.finish() + node.finish() + return dict(sorted(found.items(), key=lambda item: int(item[0].split(".")[1]))) + + +def _read_spatial(root: _Node, pair_fluents: Mapping[str, Any]) -> dict[str, dict[str, Any]] | None: + """The ``spatial`` block (F6): for a pair fluent of the file, a detector name and a mapping + of parameters. The runtime checks the form only.""" + if "spatial" not in root.data: + return None + value = root.get("spatial", None, nullable=True) + if value is None: + return None + node = _Node(root.source, root.field("spatial"), value) + if not isinstance(value, dict): + raise root.error("spatial", f"expected a mapping, found {_describe(value)}") + found: dict[str, dict[str, Any]] = {} + for label in list(value): + if not isinstance(label, str) or label not in pair_fluents: + raise node.error(label, "expected a pair fluent of the file's pair_fluents block") + entry = node.mapping(label) + detector = entry.string("detector", _MISSING) + parameters = entry.get("parameters", {}) + if not isinstance(parameters, dict): + raise entry.error("parameters", f"expected a mapping, found {_describe(parameters)}") + for key, number in parameters.items(): + if not isinstance(key, str): + raise entry.error("parameters", f"expected string keys, found {key!r}") + if isinstance(number, bool) or not isinstance(number, (int, float, str)): + raise entry.error( + f"parameters.{key}", f"expected a number or a string, found {_describe(number)}" + ) + entry.finish() + found[label] = {"detector": detector, "parameters": dict(parameters)} + return found def _read_event_type(label: str, node: _Node) -> ExplicitEventType: @@ -140,19 +231,46 @@ def _read_event_type(label: str, node: _Node) -> ExplicitEventType: return ExplicitEventType(label, requirement, requirement_text, precondition, effects, node.path) -def _parse_precondition(text: str, node: _Node) -> tuple[Literal, ...]: - """A conjunction of fluent literals and numeric threshold literals (``agent.NUMFL.1 > - 0.75``, ``agent.NUMFL.1 <= 0.75``), or TRUE.""" +def _parse_precondition(text: str, node: _Node) -> tuple[AnyLiteral, ...]: + """A conjunction of fluent literals, numeric threshold literals (``agent.NUMFL.1 > 0.75``, + ``agent.NUMFL.1 <= 0.75``), and, since stage c2a, pair literals (``PAIRFL.1(agent, + patient)``, ``NOT SOME.PAIRFL.2(agent)``), or TRUE. A pair literal's text is replaced by a + placeholder name before the expression is parsed, because its parentheses and comma are not + part of the expression grammar.""" + pairs: list[PairLiteral] = [] + + def placeholder(match: re.Match) -> str: + try: + literal = PairLiteral.match(match.group(0)) + except ValueError as error: + raise node.error("precondition", str(error)) from None + assert literal is not None + pairs.append(literal) + return f"{_PLACEHOLDER}{len(pairs) - 1}" + + substituted = _PAIR_TEXT.sub(placeholder, text) try: - expression = parse_expression(text) + expression = parse_expression(substituted) except ExpressionError as error: raise node.error("precondition", str(error)) from None - literals: list[Literal] = [] + literals: list[AnyLiteral] = [] + + def pair_literal(name: str, value: bool) -> PairLiteral: + literal = pairs[int(name[len(_PLACEHOLDER) :])] + return literal if value else literal.negated() def visit(expr: Expr) -> None: if isinstance(expr, Const) and expr.value: return - if isinstance(expr, Var): + if isinstance(expr, Var) and expr.name.startswith(_PLACEHOLDER): + literals.append(pair_literal(expr.name, True)) + elif ( + isinstance(expr, Not) + and isinstance(expr.operand, Var) + and expr.operand.name.startswith(_PLACEHOLDER) + ): + literals.append(pair_literal(expr.operand.name, False)) + elif isinstance(expr, Var): literals.append(_literal(expr.name, True, node)) elif isinstance(expr, Not) and isinstance(expr.operand, Var): literals.append(_literal(expr.operand.name, False, node)) @@ -169,7 +287,8 @@ def visit(expr: Expr) -> None: raise node.error( "precondition", f"a precondition is a conjunction of fluent literals (agent.BOOLFL., " - f"NOT patient.BOOLFL., agent.NUMFL. > 0.75, joined by AND), got {text!r}", + f"NOT patient.BOOLFL., agent.NUMFL. > 0.75, PAIRFL.(agent, patient), " + f"SOME.PAIRFL.(agent), joined by AND), got {text!r}", ) visit(expression) diff --git a/python/semantic_world/world/event_types.py b/python/semantic_world/world/event_types.py index 4636d0f..1a5df33 100644 --- a/python/semantic_world/world/event_types.py +++ b/python/semantic_world/world/event_types.py @@ -41,15 +41,23 @@ from semantic_world.world.config import Config, EffectsConfig, PreconditionsConfig from semantic_world.world.constraints import Constraint from semantic_world.world.dynamics import ( + PAIR_KIND, + PAIR_SAME, + PAIR_SOME, + PAIR_SOMEOF, + PROFILE_ORIGIN, ROLES, + AnyLiteral, ConditionLiteral, Effect, Literal, + PairLiteral, precondition_text, ) from semantic_world.world.event_file import EventFile, ExplicitEventType from semantic_world.world.fluents import Fluents from semantic_world.world.numeric import NumericFluents, drawn, empty_numeric +from semantic_world.world.pair_fluents import PairFluent, PairFluents, empty_pair_fluents from semantic_world.world.statics import StaticWorld from semantic_world.world.streams import WorldStreams @@ -146,7 +154,8 @@ class EventType: category), in feature order.""" constraints: tuple[str, ...] """The labels of the constraints whose conjunction is the requirement.""" - precondition: tuple[Literal, ...] = () + precondition: tuple[AnyLiteral, ...] = () + """The precondition literals, the compiled literals of stage c2a (origin ``profile``) last.""" effects: tuple[Effect, ...] = () explicit: bool = False @@ -294,6 +303,109 @@ def build_event_types(statics: StaticWorld) -> EventTypes: return EventTypes(tuple(constraints), tuple(event_types), features, feature_constraints) +# --------------------------------------------------------------------------------------------- +# Pair literals and effects: what they name (stage c2a) +# --------------------------------------------------------------------------------------------- + + +def _ordered(fluent: PairFluent, first: str, second: str) -> tuple[str, ...]: + """The two roles of a pair, in the fluent's canonical order: as given, or sorted for a + symmetric pair fluent, whose two directions are one pair.""" + return tuple(sorted((first, second))) if fluent.symmetric else (first, second) + + +def pair_literal_key(literal: PairLiteral, fluent: PairFluent) -> tuple: + """What a pair literal is about, so that two literals about the same thing compare equal: + a pair (both directions of a symmetric one), the row of a role (``SOME``; ``SOMEOF`` of a + symmetric pair fluent names the same set), the column of a role (``SOMEOF``), or the rows of + two roles (``SAME``, in either order).""" + if literal.kind == PAIR_KIND: + assert literal.second is not None + return ("pair", fluent.label, _ordered(fluent, literal.first, literal.second)) + if literal.kind == PAIR_SOME: + return ("row", fluent.label, literal.first) + if literal.kind == PAIR_SOMEOF: + return ("row" if fluent.symmetric else "column", fluent.label, literal.first) + assert literal.second is not None + return ("same", fluent.label, frozenset((literal.first, literal.second))) + + +def pair_literal_names(literal: PairLiteral, fluent: PairFluent) -> set[tuple]: + """The pairs, rows, and columns a pair literal names (the rule of F5 that a condition names + no pair or row the precondition names): ``SAME`` names the rows of both roles.""" + if literal.kind == PAIR_SAME: + return {("row", fluent.label, literal.first), ("row", fluent.label, literal.second)} + return {pair_literal_key(literal, fluent)} + + +def pair_effect_key(effect: Effect, fluent: PairFluent) -> tuple: + """The pair a pair effect writes, as ``pair_literal_key`` names it.""" + assert effect.second is not None + return ("pair", fluent.label, _ordered(fluent, effect.role, effect.second)) + + +def compiled_literals(effect: Effect, fluent: PairFluent) -> list[PairLiteral]: + """The compiled precondition literals (F4) of an add effect: ``NOT PAIRFL.k(second, + first)`` for an asymmetric pair fluent, ``NOT SOME.PAIRFL.k(first)`` for a functional one, + and ``NOT SOME.PAIRFL.k(second)`` too for a symmetric functional one; none for a delete or + for a general pair fluent that is not functional. Each carries the origin ``profile``.""" + if not effect.pair or not effect.value: + return [] + assert effect.second is not None + found: list[PairLiteral] = [] + if fluent.asymmetric: + found.append( + PairLiteral(PAIR_KIND, fluent.label, effect.second, effect.role, False, PROFILE_ORIGIN) + ) + if fluent.functional: + found.append(PairLiteral(PAIR_SOME, fluent.label, effect.role, None, False, PROFILE_ORIGIN)) + if fluent.symmetric: + found.append( + PairLiteral(PAIR_SOME, fluent.label, effect.second, None, False, PROFILE_ORIGIN) + ) + return found + + +def pair_literal_achievable( + literal: PairLiteral, fluent: PairFluent, adds: set[str], deletes: set[str] +) -> bool: + """Whether a pair literal can hold in some state ("Achievable" of the specification): a + positive literal when its pair fluent has a positive initial rate or some effect adds a + pair of it; a negative one when some initial state lacks the pair (an initial rate below 1, + or a functional pair fluent, in which every other pair of a row is absent) or some effect + deletes a pair of it. ``SAME`` on a symmetric functional pair fluent is never achievable: + two entities with the same partner would give that partner two partners.""" + if literal.kind == PAIR_SAME and fluent.symmetric and fluent.functional: + return not literal.value + if literal.value: + return fluent.initial_rate > 0.0 or fluent.label in adds + return fluent.initial_rate < 1.0 or fluent.functional or fluent.label in deletes + + +def add_compiled_literals( + precondition: Sequence[AnyLiteral], effects: Sequence[Effect], pair_fluents: PairFluents +) -> tuple[AnyLiteral, ...]: + """The precondition with the compiled literals of every add effect appended, in effect + order, a literal already present (by what it names, whatever its origin) not added twice.""" + present = { + pair_literal_key(lit, pair_fluents[lit.pair_fluent]): lit.value + for lit in precondition + if isinstance(lit, PairLiteral) + } + found = list(precondition) + for effect in effects: + if not effect.pair or effect.condition: + continue + fluent = pair_fluents[effect.fluent] + for literal in compiled_literals(effect, fluent): + key = pair_literal_key(literal, fluent) + if key in present: + continue + present[key] = literal.value + found.append(literal) + return tuple(found) + + # --------------------------------------------------------------------------------------------- # Preconditions and effects # --------------------------------------------------------------------------------------------- @@ -308,7 +420,7 @@ class _Draft: own_literals: list[Literal] = field(default_factory=list) explicit: ExplicitEventType | None = None effects: tuple[Effect, ...] = () - precondition: tuple[Literal, ...] = () + precondition: tuple[AnyLiteral, ...] = () own_effect_keys: set[tuple[str, str]] = field(default_factory=set) own_literal_keys: set[tuple[str, str]] = field(default_factory=set) @@ -330,12 +442,14 @@ def __init__( derived_initial: Mapping[str, np.ndarray], statics: StaticWorld | None = None, numeric: NumericFluents | None = None, + pair_fluents: PairFluents | None = None, ) -> None: self.static = event_types self.statics = statics self.fluents = fluents self.numeric = numeric if numeric is not None else empty_numeric(0) self.numeric_labels = self.numeric.labels + self.pair_fluents = pair_fluents if pair_fluents is not None else empty_pair_fluents() self.effects_config: EffectsConfig = config.event_types.effects self.preconditions_config: PreconditionsConfig = config.event_types.preconditions self.rng_effects = streams.effects @@ -513,7 +627,20 @@ def _draw_literal( ) return None - def literal_achievable(self, literal: Literal, achievable: set[tuple[str, bool]]) -> bool: + def pair_adds_and_deletes(self) -> tuple[set[str], set[str]]: + """The pair fluents some effect adds a pair of, and those some effect deletes one of.""" + adds = {e.fluent for d in self.drafts.values() for e in d.effects if e.pair and e.value} + deletes = { + e.fluent for d in self.drafts.values() for e in d.effects if e.pair and not e.value + } + return adds, deletes + + def literal_achievable(self, literal: AnyLiteral, achievable: set[tuple[str, bool]]) -> bool: + if isinstance(literal, PairLiteral): + adds, deletes = self.pair_adds_and_deletes() + return pair_literal_achievable( + literal, self.pair_fluents[literal.pair_fluent], adds, deletes + ) if literal.numeric: assert literal.threshold is not None return self.numeric_achievable(literal.fluent, literal.value, literal.threshold) @@ -584,7 +711,7 @@ def produced(self) -> list[tuple[str, bool]]: (e.fluent, e.value) for draft in self.drafts.values() for e in draft.effects - if not e.numeric + if not e.numeric and not e.pair } return sorted(values) @@ -663,7 +790,9 @@ def fix_empty_effects(self, draft: _Draft) -> bool: nothing: an own effect is redrawn, an inherited one dropped. Returns whether anything changed.""" required = { - (lit.role, lit.fluent): lit.value for lit in draft.precondition if not lit.numeric + (lit.role, lit.fluent): lit.value + for lit in draft.precondition + if not lit.numeric and not lit.pair } changed = False kept: dict[tuple[str, str], Effect] = {(e.role, e.fluent): e for e in draft.effects} @@ -701,14 +830,12 @@ def fix_unachievable( self, draft: _Draft, achievable: set[tuple[str, bool]], produced: list[tuple[str, bool]] ) -> bool: changed = False - kept: dict[tuple[str, str], Literal] = { - (lit.role, lit.fluent): lit for lit in draft.precondition - } + kept: dict[tuple, AnyLiteral] = {_literal_slot(lit): lit for lit in draft.precondition} for literal in list(draft.precondition): if self.literal_achievable(literal, achievable): continue changed = True - key = (literal.role, literal.fluent) + key = _literal_slot(literal) del kept[key] if key not in draft.own_literal_keys: self.stats["literals_dropped_unachievable"] += 1 @@ -747,6 +874,11 @@ def apply_event_file(self) -> None: f"{label} is a category of event types; give the entry to a leaf", ) for lit in entry.precondition or (): + if isinstance(lit, PairLiteral): + self._check_explicit_pair_literal( + source, f"{entry.field}.precondition", et, lit + ) + continue self._check_role(source, f"{entry.field}.precondition", et, lit.role, lit.key) if lit.numeric: if lit.fluent not in self.numeric: @@ -762,10 +894,14 @@ def apply_event_file(self) -> None: raise ConfigError( source, f"{entry.field}.precondition", f"reads unknown fluent {lit.fluent}" ) - seen_effects: dict[tuple[str, str], bool | float] = {} + seen_effects: dict[tuple, bool | float] = {} for i, effect in enumerate(entry.effects or ()): where = f"{entry.field}.effects[{i}]" self._check_role(source, where, et, effect.role, effect.key) + if effect.pair: + self._check_explicit_pair_effect(source, where, et, effect, seen_effects) + self._check_explicit_condition(source, where, et, effect) + continue if effect.numeric: if effect.fluent not in self.numeric: raise ConfigError( @@ -787,9 +923,9 @@ def apply_event_file(self) -> None: ) seen_effects[key] = effect.value self._check_explicit_condition(source, where, et, effect) - seen_literals: dict[tuple[str, str], bool] = {} + seen_literals: dict[tuple, bool] = {} for lit in entry.precondition or (): - key = (lit.role, lit.fluent) + key = _literal_slot(lit, self.pair_fluents) if key in seen_literals and (lit.numeric or seen_literals[key] != lit.value): raise ConfigError( source, @@ -798,10 +934,24 @@ def apply_event_file(self) -> None: if lit.numeric else f"holds {lit.key} and its negation", ) + if key in seen_literals: + raise ConfigError( + source, f"{entry.field}.precondition", f"holds {lit.key} twice" + ) seen_literals[key] = lit.value if entry.precondition is not None and entry.effects is not None: for effect in entry.effects: - if ( + if effect.pair: + slot = pair_effect_key(effect, self.pair_fluents[effect.fluent]) + if seen_literals.get(slot) == effect.value: + raise ConfigError( + source, + entry.field, + f"the effect {effect.text} {'adds' if effect.value else 'deletes'} " + f"a pair whose {'presence' if effect.value else 'absence'} the " + f"precondition already requires, so it changes nothing", + ) + elif ( not effect.numeric and seen_literals.get((effect.role, effect.fluent)) == effect.value ): @@ -812,7 +962,16 @@ def apply_event_file(self) -> None: f"already requires, so it changes nothing", ) for lit in effect.condition: - if lit.kind != "feature" and (lit.role, lit.symbol) in seen_literals: + if isinstance(lit, PairLiteral): + self._check_pair_condition_against( + source, + entry.field, + effect, + lit, + entry.precondition, + "already", + ) + elif lit.kind != "feature" and (lit.role, lit.symbol) in seen_literals: raise ConfigError( source, entry.field, @@ -827,8 +986,23 @@ def _check_explicit_condition( """The rules of a condition that need no other draw: every literal names a role of the event type and a known symbol, no symbol twice, and no literal on the effect's own fluent with its value (the effect would change nothing).""" - keys: set[tuple[str, str]] = set() + keys: set[tuple] = set() for lit in effect.condition: + if isinstance(lit, PairLiteral): + self._check_explicit_pair_literal(source, where, et, lit) + fluent = self.pair_fluents[lit.pair_fluent] + key = pair_literal_key(lit, fluent) + if key in keys: + raise ConfigError(source, where, f"the condition reads {lit.key} twice") + keys.add(key) + if effect.pair and key == pair_effect_key(effect, self.pair_fluents[effect.fluent]): + raise ConfigError( + source, + where, + f"the condition reads {lit.key}, the pair the effect writes: the effect " + f"would change the pair, or leave it as it is, either way", + ) + continue self._check_role(source, where, et, lit.role, lit.key) if lit.kind == "numeric": if lit.symbol not in self.numeric: @@ -851,6 +1025,151 @@ def _check_explicit_condition( f"the fluent has the effect's value after the event either way", ) + def _check_explicit_pair_literal( + self, source: str, where: str, et: EventType, lit: PairLiteral + ) -> None: + """A pair literal of an explicit entry: a known pair fluent, roles of the event type, + and ``SAME`` on a functional pair fluent only.""" + if lit.pair_fluent not in self.pair_fluents: + raise ConfigError( + source, + where, + f"{lit.key} reads unknown pair fluent {lit.pair_fluent}; the event file's " + f"pair_fluents block defines " + f"{', '.join(self.pair_fluents.labels) or 'no pair fluent'}", + ) + for role in lit.roles: + self._check_role(source, where, et, role, lit.key) + fluent = self.pair_fluents[lit.pair_fluent] + if lit.kind == PAIR_SAME and not fluent.functional: + raise ConfigError( + source, + where, + f"{lit.key}: SAME needs a functional pair fluent, and {fluent.label} is not", + ) + + def _check_explicit_pair_effect( + self, + source: str, + where: str, + et: EventType, + effect: Effect, + seen: dict[tuple, bool | float], + ) -> None: + """A pair effect of an explicit entry: a known pair fluent, roles of the event type, no + delete on a total pair fluent, no conditional add on an asymmetric or functional pair + fluent (its compiled literals would not be a plain precondition, F4), and no second + effect on the same pair (both directions of a symmetric one).""" + if effect.fluent not in self.pair_fluents: + raise ConfigError( + source, + where, + f"writes unknown pair fluent {effect.fluent}; the event file's pair_fluents " + f"block defines {', '.join(self.pair_fluents.labels) or 'no pair fluent'}", + ) + assert effect.second is not None + self._check_role(source, where, et, effect.second, effect.key) + fluent = self.pair_fluents[effect.fluent] + if fluent.total and not effect.value: + raise ConfigError( + source, + where, + f"{effect.text} deletes a pair of the total pair fluent {fluent.label}", + ) + if effect.condition and effect.value and (fluent.asymmetric or fluent.functional): + raise ConfigError( + source, + where, + f"{effect.text} is a conditional add on the " + f"{'asymmetric' if fluent.asymmetric else 'functional'} pair fluent " + f"{fluent.label}; its compiled literals would not be a plain precondition, so a " + f"conditional add needs a general pair fluent that is not functional", + ) + key = pair_effect_key(effect, fluent) + if key in seen: + raise ConfigError( + source, + where, + f"contradicts an earlier effect on {effect.key}" + if seen[key] != effect.value + else f"writes {effect.key} twice" + + ( + " (both directions of a symmetric pair are one pair)" + if fluent.symmetric + else "" + ), + ) + seen[key] = effect.value + + def _check_pair_condition_against( + self, + source: str, + where: str, + effect: Effect, + lit: PairLiteral, + precondition: Sequence[AnyLiteral], + word: str, + ) -> None: + """The rule of F5: a pair condition literal names no pair or row that the precondition + names (compiled literals included).""" + fluent = self.pair_fluents[lit.pair_fluent] + names = pair_literal_names(lit, fluent) + for other in precondition: + if not isinstance(other, PairLiteral): + continue + shared = names & pair_literal_names(other, self.pair_fluents[other.pair_fluent]) + if shared: + raise ConfigError( + source, + where, + f"the condition of {effect.text} reads {lit.key}, which names a " + f"{'row' if shared and next(iter(shared))[0] == 'row' else 'pair or column'} " + f"the precondition {word} names ({other.key})", + ) + + def compile_profile_literals(self) -> None: + """Append the compiled literals (F4) to every event type with an add effect on a pair + fluent; an explicit entry whose compiled literals contradict its own precondition is an + error naming the file and the entry.""" + for draft in self.drafts.values(): + if not any(e.pair for e in draft.effects): + continue + before = draft.precondition + keys = { + pair_literal_key(lit, self.pair_fluents[lit.pair_fluent]): lit + for lit in before + if isinstance(lit, PairLiteral) + } + for effect in draft.effects: + if not effect.pair or effect.condition or not effect.value: + continue + fluent = self.pair_fluents[effect.fluent] + for literal in compiled_literals(effect, fluent): + other = keys.get(pair_literal_key(literal, fluent)) + if other is not None and other.value != literal.value: + assert self.event_file is not None and draft.explicit is not None + raise ConfigError( + self.event_file.source, + draft.explicit.field, + f"the effect {effect.text} needs the compiled literal {literal.text} " + f"to keep the profile of {fluent.label}, which contradicts the " + f"precondition literal {other.text}: the event type could never " + f"happen", + ) + draft.precondition = add_compiled_literals(before, draft.effects, self.pair_fluents) + if draft.explicit is not None and draft.explicit.effects is not None: + for effect in draft.effects: + for lit in effect.condition: + if isinstance(lit, PairLiteral): + self._check_pair_condition_against( + self.event_file.source, # type: ignore[union-attr] + f"{draft.explicit.field}.effects", + effect, + lit, + draft.precondition, + "or its compiled literals", + ) + def _check_threshold( self, source: str, where: str, fluent: str, threshold: float | None ) -> None: @@ -895,8 +1214,10 @@ def check_explicit_achievable(self) -> None: f"the condition literal {lit.text} is not achievable: no entity " f"starts with that value and no effect or drift produces it", ) + if isinstance(lit, PairLiteral): + continue if lit.kind != "feature" and (lit.role, lit.symbol) in { - (p.role, p.fluent) for p in draft.precondition + (p.role, p.fluent) for p in draft.precondition if not p.pair }: raise ConfigError( self.event_file.source, @@ -914,8 +1235,10 @@ def check_explicit_achievable(self) -> None: ) def _condition_literal_achievable( - self, lit: ConditionLiteral, achievable: set[tuple[str, bool]] + self, lit: ConditionLiteral | PairLiteral, achievable: set[tuple[str, bool]] ) -> bool: + if isinstance(lit, PairLiteral): + return self.literal_achievable(lit, achievable) if lit.kind == "feature": return True if lit.numeric: @@ -1141,6 +1464,7 @@ def run(self) -> EventTypes: changed |= self.fix_unachievable(draft, achievable, produced) if not changed: break + self.compile_profile_literals() self.draw_conditions() self.check_explicit_achievable() event_types = tuple( @@ -1178,18 +1502,39 @@ def generate_event_types( event_file: EventFile | None, derived_initial: Mapping[str, np.ndarray], numeric: NumericFluents | None = None, + pair_fluents: PairFluents | None = None, ) -> EventTypes: """Event types with their requirements, preconditions, and effects.""" static = build_event_types(statics) return _Dynamics( - static, fluents, config, streams, event_file, derived_initial, statics, numeric + static, + fluents, + config, + streams, + event_file, + derived_initial, + statics, + numeric, + pair_fluents, ).run() +def _literal_slot(literal: AnyLiteral, pair_fluents: PairFluents | None = None) -> tuple: + """The slot a precondition literal fills: ``(role, fluent)`` for a Boolean or numeric + literal, and what the literal names for a pair literal (with the pair fluents, so that both + directions of a symmetric pair are one slot; without them, the literal's key).""" + if isinstance(literal, PairLiteral): + if pair_fluents is not None and literal.pair_fluent in pair_fluents: + return pair_literal_key(literal, pair_fluents[literal.pair_fluent]) + return ("pair_literal", literal.key) + return (literal.role, literal.fluent) + + def condition_keys(effects: Sequence[Effect]) -> set[tuple[str, str]]: """The ``(role, fluent)`` pairs that the conditions of the effects read (Boolean or numeric): a precondition literal on one of them would fix what the condition asks, so none - is drawn there.""" + is drawn there. A pair condition literal (stage c2a) names its pair fluent, which no Boolean + literal can name.""" return { (lit.role, lit.symbol) for effect in effects @@ -1259,9 +1604,13 @@ def ok(literal: Literal) -> bool: literal = feature_literals.get(feature) if literal is not None and feature in et.features and ok(literal): inherited.setdefault((literal.role, literal.fluent), literal) - empty = {(e.role, e.fluent, e.value) for e in et.effects if not e.numeric} + empty = {(e.role, e.fluent, e.value) for e in et.effects if not e.numeric and not e.pair} read = condition_keys(et.effects) - kept = dict(inherited) + kept: dict[tuple, AnyLiteral] = dict(inherited) + # The compiled literals of stage c2a (origin profile) stay with the event type. + for lit in et.precondition: + if isinstance(lit, PairLiteral) and lit.compiled: + kept[("compiled", lit.key)] = lit n = _draw(rng, cfg.literals) for _ in range(n): for _ in range(MAX_TRIES): @@ -1337,6 +1686,7 @@ def check_dynamics( initial_present: set[tuple[str, bool]], features: Sequence[str] | None = None, numeric: NumericFluents | None = None, + pair_fluents: PairFluents | None = None, ) -> None: """The invariants of "Preconditions", "Effects", and "Conditional effects": no effect on a derived fluent, no contradiction (whatever the conditions), no empty effect, every literal @@ -1347,11 +1697,19 @@ def check_dynamics( otherwise. Tests and the generator both call it. Whether some able binding satisfies the static literals needs the static side; the generator checks it when it draws.""" produced = { - (e.fluent, e.value) for et in event_types.event_types for e in et.effects if not e.numeric + (e.fluent, e.value) + for et in event_types.event_types + for e in et.effects + if not e.numeric and not e.pair } achievable = produced | initial_present signs = produced_numeric(event_types) numeric = numeric if numeric is not None else empty_numeric(0) + pair_fluents = pair_fluents if pair_fluents is not None else empty_pair_fluents() + adds = {e.fluent for et in event_types.event_types for e in et.effects if e.pair and e.value} + deletes = { + e.fluent for et in event_types.event_types for e in et.effects if e.pair and not e.value + } def numeric_ok(fluent: str, above: bool, threshold: float | None) -> bool: return ( @@ -1360,12 +1718,49 @@ def numeric_ok(fluent: str, above: bool, threshold: float | None) -> bool: and numeric.achievable(fluent, above, threshold, signs) ) + def pair_ok(lit: PairLiteral) -> bool: + return lit.pair_fluent in pair_fluents and pair_literal_achievable( + lit, pair_fluents[lit.pair_fluent], adds, deletes + ) + for et in event_types.event_types: - effect_keys: dict[tuple[str, str], bool | float] = {} - precondition_keys = {(lit.role, lit.fluent) for lit in et.precondition} + effect_keys: dict[tuple, bool | float] = {} + precondition_keys = {(lit.role, lit.fluent) for lit in et.precondition if not lit.pair} + pair_precondition: dict[tuple, PairLiteral] = {} + pair_names: set[tuple] = set() + for lit in et.precondition: + if not isinstance(lit, PairLiteral): + continue + if lit.pair_fluent not in pair_fluents: + raise ValueError(f"{et.label}: the literal {lit.text} reads an unknown pair fluent") + fluent = pair_fluents[lit.pair_fluent] + for role in lit.roles: + if role not in et.roles: + raise ValueError(f"{et.label}: the literal {lit.text} uses a role it has not") + if lit.kind == PAIR_SAME and not fluent.functional: + raise ValueError( + f"{et.label}: the literal {lit.text} reads SAME on a pair fluent that is not " + f"functional" + ) + key = pair_literal_key(lit, fluent) + if key in pair_precondition: + raise ValueError( + f"{et.label}: the precondition reads {lit.key} twice" + if pair_precondition[key].value == lit.value + else f"{et.label}: the precondition holds {lit.key} and its negation" + ) + pair_precondition[key] = lit + pair_names |= pair_literal_names(lit, fluent) + if not pair_ok(lit): + raise ValueError(f"{et.label}: the literal {lit.text} is not achievable") for effect in et.effects: if effect.role not in et.roles: raise ValueError(f"{et.label}: the effect {effect.text} uses a role it has not") + if effect.pair: + _check_pair_effect( + et, effect, pair_fluents, effect_keys, pair_precondition, pair_names, pair_ok + ) + continue if effect.numeric: if effect.fluent not in numeric: raise ValueError( @@ -1379,8 +1774,13 @@ def numeric_ok(fluent: str, above: bool, threshold: float | None) -> bool: if key in effect_keys: raise ValueError(f"{et.label}: two effects write {effect.key}") effect_keys[key] = effect.value - seen: set[tuple[str, str]] = set() + seen: set[tuple] = set() for lit in effect.condition: + if isinstance(lit, PairLiteral): + _check_pair_condition_literal( + et, effect, lit, pair_fluents, pair_names, seen, pair_ok + ) + continue if lit.role not in et.roles: raise ValueError( f"{et.label}: the condition of {effect.text} uses a role it has not" @@ -1426,6 +1826,8 @@ def numeric_ok(fluent: str, above: bool, threshold: float | None) -> bool: seen.add((lit.role, lit.symbol)) literal_keys: dict[tuple[str, str], bool] = {} for lit in et.precondition: + if lit.pair: + continue if lit.role not in et.roles: raise ValueError(f"{et.label}: the literal {lit.text} uses a role it has not") key = (lit.role, lit.fluent) @@ -1441,12 +1843,113 @@ def numeric_ok(fluent: str, above: bool, threshold: float | None) -> bool: if (lit.fluent, lit.value) not in achievable: raise ValueError(f"{et.label}: the literal {lit.text} is not achievable") for key, value in effect_keys.items(): + if key[0] == "pair": + continue if not key[1].startswith("NUMFL.") and literal_keys.get(key) == value: raise ValueError(f"{et.label}: the effect on {key[0]}.{key[1]} changes nothing") +def _check_pair_effect( + et: EventType, + effect: Effect, + pair_fluents: PairFluents, + effect_keys: dict[tuple, bool | float], + pair_precondition: Mapping[tuple, PairLiteral], + pair_names: set[tuple], + pair_ok, +) -> None: + """The invariants of a pair effect (stage c2a): a known pair fluent, roles of the event + type, no delete on a total pair fluent, no second effect on the same pair (both directions + of a symmetric one), no add whose precondition requires the pair and no delete whose + precondition requires its absence, no conditional add on an asymmetric or functional pair + fluent, the compiled literals of an add present in the precondition, and every pair + condition literal well formed.""" + if effect.fluent not in pair_fluents: + raise ValueError(f"{et.label}: the effect {effect.text} writes an unknown pair fluent") + fluent = pair_fluents[effect.fluent] + assert effect.second is not None + if effect.second not in et.roles: + raise ValueError(f"{et.label}: the effect {effect.text} uses a role it has not") + if fluent.total and not effect.value: + raise ValueError( + f"{et.label}: the effect {effect.text} deletes a pair of a total pair fluent" + ) + key = pair_effect_key(effect, fluent) + if key in effect_keys: + raise ValueError(f"{et.label}: two effects write {effect.key}") + effect_keys[key] = effect.value + required = pair_precondition.get(key) + if required is not None and required.value == effect.value: + raise ValueError(f"{et.label}: the effect {effect.text} changes nothing") + if effect.condition and effect.value and (fluent.asymmetric or fluent.functional): + raise ValueError( + f"{et.label}: the effect {effect.text} is a conditional add on an asymmetric or " + f"functional pair fluent" + ) + if not effect.condition: + for literal in compiled_literals(effect, fluent): + present = pair_precondition.get(pair_literal_key(literal, fluent)) + if present is None or present.value != literal.value: + raise ValueError( + f"{et.label}: the effect {effect.text} lacks its compiled literal " + f"{literal.text}" + ) + seen: set[tuple] = set() + for lit in effect.condition: + if isinstance(lit, PairLiteral): + _check_pair_condition_literal(et, effect, lit, pair_fluents, pair_names, seen, pair_ok) + elif lit.role not in et.roles: + raise ValueError(f"{et.label}: the condition of {effect.text} uses a role it has not") + + +def _check_pair_condition_literal( + et: EventType, + effect: Effect, + lit: PairLiteral, + pair_fluents: PairFluents, + pair_names: set[tuple], + seen: set[tuple], + pair_ok, +) -> None: + """The invariants of a pair condition literal (F5): a known pair fluent, roles of the event + type, ``SAME`` on a functional pair fluent, achievable, no literal twice, naming no pair or + row the precondition names, and not the pair the effect itself writes.""" + if lit.pair_fluent not in pair_fluents: + raise ValueError(f"{et.label}: the condition of {effect.text} reads an unknown pair fluent") + fluent = pair_fluents[lit.pair_fluent] + for role in lit.roles: + if role not in et.roles: + raise ValueError(f"{et.label}: the condition of {effect.text} uses a role it has not") + if lit.kind == PAIR_SAME and not fluent.functional: + raise ValueError( + f"{et.label}: the condition literal {lit.text} reads SAME on a pair fluent that is " + f"not functional" + ) + if not pair_ok(lit): + raise ValueError(f"{et.label}: the condition literal {lit.text} is not achievable") + key = pair_literal_key(lit, fluent) + if key in seen: + raise ValueError(f"{et.label}: the condition of {effect.text} reads {lit.key} twice") + seen.add(key) + if pair_literal_names(lit, fluent) & pair_names: + raise ValueError( + f"{et.label}: the condition of {effect.text} reads {lit.key}, which names a pair or " + f"a row the precondition names" + ) + if effect.pair and key == pair_effect_key(effect, pair_fluents[effect.fluent]): + raise ValueError( + f"{et.label}: the condition of {effect.text} reads the pair the effect writes" + ) + + __all__ = [ "CONSTRAINT_PREFIX", + "add_compiled_literals", + "compiled_literals", + "pair_effect_key", + "pair_literal_achievable", + "pair_literal_key", + "pair_literal_names", "FEATURE_PREFIX", "ONE_PLACE", "REQUIREMENT_PREFIX", diff --git a/python/semantic_world/world/fixtures.py b/python/semantic_world/world/fixtures.py index a2e295d..b70dcbf 100644 --- a/python/semantic_world/world/fixtures.py +++ b/python/semantic_world/world/fixtures.py @@ -14,6 +14,12 @@ order ``clamp(clamp(v + e) + d)`` with plain Python floats, and judges the comparisons of "Phase (b): the world" by direct arithmetic; a fixture may give ``initial_numeric``, a ``numeric`` record after each step, and a list of ``comparisons`` with their expected values. +Since stage c2a a state has a pair part (``{pair fluent: frozenset of (first, second) label +pairs}``), which the evaluator reads directly, never through the matrices: the pair literals of +rules, preconditions, and conditions; the pair effects, both directions of a symmetric pair; +the pairs, rows, and columns an event reads for non-interference. A fixture of a world with pair +fluents gives ``initial_pairs`` beside ``initial`` and ``pairs`` after each step, as ``[pair +fluent, first, second]`` triples. """ from __future__ import annotations @@ -36,6 +42,7 @@ from semantic_world.world.errors import StepError, WorldError from semantic_world.world.identity import read_json, rule_set_id, to_json, write_json from semantic_world.world.numeric import clamp +from semantic_world.world.pair_fluents import initial_pairs_frame from semantic_world.world.places import places_frame from semantic_world.world.runtime import ( ENTITY, @@ -67,6 +74,10 @@ """A state for the evaluator: each entity's set of true base fluents.""" BruteNumeric = dict[str, dict[str, float]] """The numeric part of a state: each entity's value of each numeric fluent.""" +BrutePairs = dict[str, frozenset[tuple[str, str]]] +"""The pair part of a state (stage c2a): for each pair fluent, the ordered pairs that hold, by +labels; both directions of a symmetric pair.""" +PAIR_KINDS = ("pair", "pair_some", "pair_someof", "pair_same") @dataclass(frozen=True) @@ -129,8 +140,12 @@ def __init__( if s["kind"] == "numeric_fluent" } self.scalars = [s["label"] for s in record["symbols"] if s["kind"] == "scalar"] + self.pair_fluents = [s["label"] for s in record["symbols"] if s["kind"] == "pair_fluent"] + self.pair_profile = {s["label"]: s for s in record["symbols"] if s["kind"] == "pair_fluent"} self._numeric: BruteNumeric | None = None """The numeric part of the state being evaluated (set by the operations).""" + self._pairs: BrutePairs | None = None + """The pair part of the state being evaluated (stage c2a; set by the operations).""" # Rules and literals --------------------------------------------------------------------- @@ -170,8 +185,47 @@ def literal( return self.fluent(literal["fluent"], entity, state) if kind == "numeric_threshold": return int(self._value(entity, literal["fluent"]) > literal["threshold"]) + if kind in PAIR_KINDS: + return int(self.pair_literal(literal, roles)) raise WorldError(f"unknown literal kind {kind!r}") + # Pairs (stage c2a) ---------------------------------------------------------------------- + + def _pair_set(self, fluent: str) -> frozenset[tuple[str, str]]: + if self._pairs is None: + raise WorldError(f"the evaluator reads {fluent} without the state's pairs") + return self._pairs[fluent] + + def symmetric(self, fluent: str) -> bool: + return self.pair_profile[fluent]["symmetry"] == "symmetric" + + def pair_literal(self, literal: Mapping[str, Any], roles: Mapping[str, str]) -> bool: + """A pair literal of a rule (``role`` null: the entity itself), a precondition, or a + condition, read directly from the pair sets: ``pair`` holds when the ordered pair is in + the set (a symmetric pair fluent holds both directions); ``pair_some`` when some pair + starts at the entity; ``pair_someof`` when some pair ends at it; ``pair_same`` when the + two entities have a common partner. The literal's ``value`` (absent in the literal + table, where it is the positive literal) negates it when false.""" + pairs = self._pair_set(literal["pair_fluent"]) + kind = literal["kind"] + if "role" in literal and "first" not in literal: + first = roles[literal["role"] if literal["role"] is not None else "entity"] + else: + first = roles[literal["first"]] + if kind == "pair": + positive = (first, roles[literal["second"]]) in pairs + elif kind == "pair_some": + positive = any(a == first for a, _ in pairs) + elif kind == "pair_someof": + positive = any(b == first for _, b in pairs) + else: + second = roles[literal["second"]] + partners_of_first = {b for a, b in pairs if a == first} + partners_of_second = {b for a, b in pairs if a == second} + positive = bool(partners_of_first & partners_of_second) + value = literal.get("value", True) + return positive if value else not positive + def _row(self, label: str) -> Mapping[str, Any]: """The static facts of an entity, or of a place.""" row = self.entities.get(label) @@ -199,15 +253,28 @@ def with_numeric(self, numeric: BruteNumeric | None) -> BruteForce: self._numeric = numeric return self + def with_pairs(self, pairs: BrutePairs | None) -> BruteForce: + """Set the pair part of the state the next operations evaluate (None keeps the one set + before; stage c2a).""" + if pairs is not None: + self._pairs = pairs + return self + def derived_static(self, entity: str) -> list[str]: return [f for f in self.derived_features if self.feature(f, entity)] def derived_fluent( - self, entity: str, state: BruteState, numeric: BruteNumeric | None = None + self, + entity: str, + state: BruteState, + numeric: BruteNumeric | None = None, + pairs: BrutePairs | None = None, ) -> list[str]: - self.with_numeric(numeric) + self.with_numeric(numeric).with_pairs(pairs) if self.numeric_fluents and self._numeric is None: raise WorldError("the world has numeric fluents, so an operation needs their values") + if self.pair_fluents and self._pairs is None: + raise WorldError("the world has pair fluents, so an operation needs their pairs") return [f for f in self.derived_fluents if self.fluent(f, entity, state)] def able(self, event: BruteEvent) -> bool: @@ -218,11 +285,19 @@ def _threshold_holds(self, lit: Mapping[str, Any], value: float) -> bool: return value > lit["threshold"] if lit["operator"] == ">" else value <= lit["threshold"] def precondition_holds( - self, event: BruteEvent, state: BruteState, numeric: BruteNumeric | None = None + self, + event: BruteEvent, + state: BruteState, + numeric: BruteNumeric | None = None, + pairs: BrutePairs | None = None, ) -> bool: - self.with_numeric(numeric) + self.with_numeric(numeric).with_pairs(pairs) et = self.event_types[event.event_type] for lit in et["precondition"]["literals"]: + if lit.get("kind") in PAIR_KINDS: + if not self.pair_literal(lit, event.binding): + return False + continue entity = event.binding[lit["role"]] if "threshold" in lit: if not self._threshold_holds(lit, self._value(entity, lit["fluent"])): @@ -234,9 +309,13 @@ def precondition_holds( return True def legal( - self, event: BruteEvent, state: BruteState, numeric: BruteNumeric | None = None + self, + event: BruteEvent, + state: BruteState, + numeric: BruteNumeric | None = None, + pairs: BrutePairs | None = None, ) -> bool: - return self.able(event) and self.precondition_holds(event, state, numeric) + return self.able(event) and self.precondition_holds(event, state, numeric, pairs) def bindings( self, event_type: str, entities: Sequence[str] | None = None @@ -257,11 +336,13 @@ def legal_bindings( state: BruteState, entities: Sequence[str] | None = None, numeric: BruteNumeric | None = None, + pairs: BrutePairs | None = None, ) -> list[dict[str, str]]: + self.with_numeric(numeric).with_pairs(pairs) return [ b for b in self.bindings(event_type, entities) - if self.legal(BruteEvent(event_type, b), state, numeric) + if self.legal(BruteEvent(event_type, b), state) ] def performable(self) -> list[str]: @@ -279,6 +360,23 @@ def cone(self, fluent: str) -> set[str]: found |= self.cone(literal["fluent"]) return found + def pair_cone(self, fluent: str) -> set[tuple[str, str]]: + """The rows and columns of its own entity a fluent reads through the pair literals of + its rule, directly or through other derived fluents: ``("row", pair fluent)`` for + ``pair_some`` and ``("column", pair fluent)`` for ``pair_someof``.""" + if not self.symbols[fluent]["derived"]: + return set() + found: set[tuple[str, str]] = set() + for i in self.rules[fluent]["inputs"]: + literal = self.literals[i] + if literal["kind"] == "pair_some": + found.add(("row", literal["pair_fluent"])) + elif literal["kind"] == "pair_someof": + found.add(("column", literal["pair_fluent"])) + elif literal["kind"] in ("fluent", "numeric_threshold"): + found |= self.pair_cone(literal["fluent"]) + return found + @staticmethod def _effect_value(effect: Mapping[str, Any]) -> bool | float: """A Boolean effect's value, or a numeric effect's signed constant.""" @@ -287,12 +385,31 @@ def _effect_value(effect: Mapping[str, Any]) -> bool | float: return amount if effect["operator"] == "+=" else -amount return bool(effect["value"]) - def writes(self, event: BruteEvent) -> dict[tuple[str, str], bool | float]: - """What the event may write: every effect, whatever its condition.""" + def _pair_effect_keys( + self, effect: Mapping[str, Any], binding: Mapping[str, str] + ) -> list[tuple[str, str, str]]: + """The ordered pairs a pair effect writes: the pair between its roles, and its reverse + for a symmetric pair fluent, as ``(pair fluent, first, second)``.""" + fluent = effect["pair_fluent"] + first, second = binding[effect["first"]], binding[effect["second"]] + keys = [(fluent, first, second)] + if self.symmetric(fluent): + keys.append((fluent, second, first)) + return keys + + def writes(self, event: BruteEvent) -> dict[tuple, bool | float]: + """What the event may write: every effect, whatever its condition: ``(entity, + fluent)`` for a Boolean or numeric fluent, ``(pair fluent, first, second)`` for a pair + (both directions of a symmetric one).""" et = self.event_types[event.event_type] - return { - (event.binding[e["role"]], e["fluent"]): self._effect_value(e) for e in et["effects"] - } + found: dict[tuple, bool | float] = {} + for e in et["effects"]: + if e.get("kind") == "pair": + for key in self._pair_effect_keys(e, event.binding): + found[key] = bool(e["value"]) + else: + found[(event.binding[e["role"]], e["fluent"])] = self._effect_value(e) + return found def condition_holds( self, @@ -308,6 +425,10 @@ def condition_holds( if numeric is not None: self.with_numeric(numeric) for lit in (effect.get("condition") or {}).get("literals", ()): + if lit["kind"] in PAIR_KINDS: + if not self.pair_literal(lit, binding): + return False + continue entity = binding[lit["role"]] if lit["kind"] == "numeric": if not self._threshold_holds(lit, self._value(entity, lit["symbol"])): @@ -322,32 +443,85 @@ def condition_holds( return True def fired( - self, event: BruteEvent, state: BruteState, numeric: BruteNumeric | None = None - ) -> dict[tuple[str, str], bool | float]: + self, + event: BruteEvent, + state: BruteState, + numeric: BruteNumeric | None = None, + pairs: BrutePairs | None = None, + ) -> dict[tuple, bool | float]: """What the event writes in the state: its effects whose conditions hold there.""" - self.with_numeric(numeric) + self.with_numeric(numeric).with_pairs(pairs) et = self.event_types[event.event_type] - return { - (event.binding[e["role"]], e["fluent"]): self._effect_value(e) - for e in et["effects"] - if self.condition_holds(e, event.binding, state) - } + found: dict[tuple, bool | float] = {} + for e in et["effects"]: + if not self.condition_holds(e, event.binding, state): + continue + if e.get("kind") == "pair": + for key in self._pair_effect_keys(e, event.binding): + found[key] = bool(e["value"]) + else: + found[(event.binding[e["role"]], e["fluent"])] = self._effect_value(e) + return found + + def _row_pairs(self, fluent: str, entity: str) -> set[tuple[str, str, str]]: + return {(fluent, entity, z) for z in self.entity_labels if z != entity} + + def _column_pairs(self, fluent: str, entity: str) -> set[tuple[str, str, str]]: + return {(fluent, z, entity) for z in self.entity_labels if z != entity} + + def _pair_literal_reads( + self, lit: Mapping[str, Any], binding: Mapping[str, str] + ) -> set[tuple[str, str, str]]: + """The ordered pairs a pair literal reads, every pair of a row or a column counted: the + pair (both directions of a symmetric one) for ``pair``, the row for ``pair_some``, the + column for ``pair_someof``, both rows for ``pair_same``.""" + fluent = lit["pair_fluent"] + if lit["kind"] == "pair": + first, second = binding[lit["first"]], binding[lit["second"]] + found = {(fluent, first, second)} + if self.symmetric(fluent): + found.add((fluent, second, first)) + return found + if lit["kind"] == "pair_some": + return self._row_pairs(fluent, binding[lit["role"]]) + if lit["kind"] == "pair_someof": + return self._column_pairs(fluent, binding[lit["role"]]) + return self._row_pairs(fluent, binding[lit["first"]]) | self._row_pairs( + fluent, binding[lit["second"]] + ) + + def _fluent_reads(self, fluent: str, entity: str) -> set[tuple]: + """What a literal on a fluent of an entity reads: the base fluents of its cone, and the + rows and columns of the entity its rule reads through pair literals.""" + found: set[tuple] = {(entity, base) for base in self.cone(fluent)} + for kind, pair_fluent in self.pair_cone(fluent): + found |= ( + self._row_pairs(pair_fluent, entity) + if kind == "row" + else self._column_pairs(pair_fluent, entity) + ) + return found - def reads(self, event: BruteEvent) -> set[tuple[str, str]]: - """The base fluents (Boolean or numeric) the precondition and the effects' conditions - read, through the cones of derived fluents; a static literal reads nothing an event can + def reads(self, event: BruteEvent) -> set[tuple]: + """The facts the precondition and the effects' conditions read, through the cones of + derived fluents: base fluents (Boolean or numeric) of an entity, and ordered pairs (a + pair, or every pair of a row or a column); a static literal reads nothing an event can write.""" et = self.event_types[event.event_type] - found: set[tuple[str, str]] = set() + found: set[tuple] = set() for lit in et["precondition"]["literals"]: - for base in self.cone(lit["fluent"]): - found.add((event.binding[lit["role"]], base)) + if lit.get("kind") in PAIR_KINDS: + found |= self._pair_literal_reads(lit, event.binding) + continue + found |= self._fluent_reads(lit["fluent"], event.binding[lit["role"]]) for effect in et["effects"]: for lit in (effect.get("condition") or {}).get("literals", ()): + if lit["kind"] in PAIR_KINDS: + found |= self._pair_literal_reads(lit, event.binding) + continue if lit["kind"] == "feature": continue - for base in self.cone(lit["symbol"]): - found.add((event.binding[lit["role"]], base)) + found |= self._fluent_reads(lit["symbol"], event.binding[lit["role"]]) return found def interfere(self, a: BruteEvent, b: BruteEvent) -> bool: @@ -364,19 +538,56 @@ def initial_numeric(self) -> BruteNumeric: for e in self.entity_labels } + def empty_pairs(self) -> BrutePairs: + return {f: frozenset() for f in self.pair_fluents} + + def check_profiles(self, pairs: BrutePairs) -> str | None: + """Why the pair sets break a profile, or None: a pair of an entity with itself, a + symmetric pair without its reverse, an asymmetric pair with its reverse, or two pairs in + a row of a functional pair fluent.""" + for fluent, held in pairs.items(): + profile = self.pair_profile[fluent] + for a, b in held: + if a == b: + return f"{fluent} holds between {a} and itself" + if profile["symmetry"] == "symmetric" and (b, a) not in held: + return f"{fluent} is symmetric, but ({a}, {b}) holds without ({b}, {a})" + if profile["symmetry"] == "asymmetric" and (b, a) in held: + return f"{fluent} is asymmetric, but ({a}, {b}) and ({b}, {a}) both hold" + if profile["functional"]: + for entity in self.entity_labels: + if sum(1 for a, _ in held if a == entity) > 1: + return f"{fluent} is functional, but the row of {entity} holds two pairs" + return None + def step( - self, state: BruteState, numeric: BruteNumeric | None, events: Sequence[BruteEvent] + self, + state: BruteState, + numeric: BruteNumeric | None, + events: Sequence[BruteEvent], + pairs: BrutePairs | None = None, ) -> tuple[BruteState, BruteNumeric, BruteNumeric]: """The next state and numeric values, and the numeric values after the effects and before the drift. The caller has checked legality and non-interference. Every condition is judged in ``state``, before any effect of the step; each numeric value becomes - ``clamp(clamp(v + e) + d)``, in plain Python floats.""" + ``clamp(clamp(v + e) + d)``, in plain Python floats. The pairs after the step are read + back with ``pairs_after``.""" numeric = numeric if numeric is not None else self.initial_numeric() - self.with_numeric(numeric) + pairs = pairs if pairs is not None else (self._pairs or self.empty_pairs()) + self.with_numeric(numeric).with_pairs(pairs) new = {e: set(v) for e, v in state.items()} + new_pairs = {f: set(v) for f, v in pairs.items()} after_effects = {e: dict(v) for e, v in numeric.items()} for event in events: - for (entity, fluent), value in self.fired(event, state).items(): + for key, value in self.fired(event, state).items(): + if len(key) == 3: + fluent, first, second = key + if value: + new_pairs[fluent].add((first, second)) + else: + new_pairs[fluent].discard((first, second)) + continue + entity, fluent = key if fluent in self.numeric_range: low, high = self.numeric_range[fluent] after_effects[entity][fluent] = clamp( @@ -393,14 +604,39 @@ def step( } for e, values in after_effects.items() } + self._pairs_after: BrutePairs = {f: frozenset(v) for f, v in new_pairs.items()} + violation = self.check_profiles(self._pairs_after) + if violation is not None: + raise WorldError(f"after the step a profile does not hold: {violation}") return {e: frozenset(v) for e, v in new.items()}, final, after_effects + @property + def pairs_after(self) -> BrutePairs: + """The pairs after the last ``step``.""" + return self._pairs_after + def apply( self, state: BruteState, events: Sequence[BruteEvent], numeric: BruteNumeric | None = None ) -> BruteState: """The next Boolean state (``step`` gives the numeric values too).""" return self.step(state, numeric, events)[0] + def pairs_record(self, pairs: BrutePairs) -> list[list[str]]: + """The pairs as a fixture writes them: ``[pair fluent, first, second]`` triples, by + pair fluent in symbol order, then by first and second in entity order.""" + order = {label: i for i, label in enumerate(self.entity_labels)} + return [ + [fluent, a, b] + for fluent in self.pair_fluents + for a, b in sorted(pairs[fluent], key=lambda p: (order[p[0]], order[p[1]])) + ] + + def pairs_from_record(self, rows: Sequence[Sequence[str]]) -> BrutePairs: + found: dict[str, set[tuple[str, str]]] = {f: set() for f in self.pair_fluents} + for fluent, a, b in rows: + found[str(fluent)].add((str(a), str(b))) + return {f: frozenset(v) for f, v in found.items()} + def state_record(self, state: BruteState) -> dict[str, list[str]]: return {e: [f for f in self.base_fluents if f in state[e]] for e in self.entity_labels} @@ -410,16 +646,22 @@ def numeric_record(self, numeric: BruteNumeric) -> dict[str, dict[str, float]]: } def derived_record( - self, state: BruteState, numeric: BruteNumeric | None = None + self, + state: BruteState, + numeric: BruteNumeric | None = None, + pairs: BrutePairs | None = None, ) -> dict[str, list[str]]: - return {e: self.derived_fluent(e, state, numeric) for e in self.entity_labels} + self.with_numeric(numeric).with_pairs(pairs) + return {e: self.derived_fluent(e, state) for e in self.entity_labels} def legal_record( - self, state: BruteState, numeric: BruteNumeric | None = None + self, + state: BruteState, + numeric: BruteNumeric | None = None, + pairs: BrutePairs | None = None, ) -> dict[str, list[dict[str, str]]]: - return { - label: self.legal_bindings(label, state, None, numeric) for label in self.performable() - } + self.with_numeric(numeric).with_pairs(pairs) + return {label: self.legal_bindings(label, state) for label in self.performable()} # Comparisons ---------------------------------------------------------------------------- @@ -505,10 +747,11 @@ def fixture_record( initial_numeric: Mapping[str, Mapping[str, float]] | None = None, comparisons: Sequence[Mapping[str, Any]] = (), places: Sequence[Mapping[str, Any]] | None = None, + initial_pairs: Sequence[Sequence[str]] | None = None, ) -> dict[str, Any]: """Assemble a fixture, with the rule-set identity recomputed from the inline definition. - ``initial_numeric`` and ``comparisons`` (stage b2) and ``places`` (stage c1) are written - only when given.""" + ``initial_numeric`` and ``comparisons`` (stage b2), ``places`` (stage c1), and + ``initial_pairs`` (stage c2a) are written only when given.""" identity = rule_set_id( record["symbols"], record["literals"], record["rules"], record["event_types"] ) @@ -525,6 +768,8 @@ def fixture_record( fixture["initial"] = dict(initial) if initial_numeric is not None: fixture["initial_numeric"] = {e: dict(v) for e, v in initial_numeric.items()} + if initial_pairs is not None: + fixture["initial_pairs"] = [list(triple) for triple in initial_pairs] fixture["steps"] = list(steps) fixture["error"] = error if comparisons: @@ -604,7 +849,10 @@ def check_fixture(fixture: Mapping[str, Any]) -> FixtureReport: ) try: state = State.from_true( - definition, fixture["initial"], _initial_numeric(definition, fixture) + definition, + fixture["initial"], + _initial_numeric(definition, fixture), + _initial_pairs(definition, fixture), ) except WorldError as err: raise _fail(name, f"has a malformed initial state: {err}") from None @@ -684,6 +932,15 @@ def check_fixture(fixture: Mapping[str, Any]) -> FixtureReport: f"step {k}: the numeric values after the step are {got_numeric}, but the " f"fixture expects {step.get('numeric')}", ) + if definition.pair_fluents or "pairs" in step: + got_pairs = state.pairs_record(definition) + expected_pairs = [list(t) for t in step.get("pairs", [])] + if got_pairs != expected_pairs: + raise _fail( + name, + f"step {k}: the pairs after the step are {got_pairs}, but the fixture " + f"expects {expected_pairs}", + ) states.append(state) for entry in fixture.get("comparisons", ()): got = _runtime_comparison(definition, entry, states) @@ -707,6 +964,18 @@ def _initial_numeric( return {e: {f: float(x) for f, x in v.items()} for e, v in given.items()} +def _initial_pairs( + definition: RuntimeDefinition, fixture: Mapping[str, Any] +) -> list[list[str]] | None: + """The fixture's ``initial_pairs``, or None (the definition's initial pairs) when it has + none and the world has no pair fluents; a world with pair fluents and no ``initial_pairs`` + starts with none.""" + given = fixture.get("initial_pairs") + if given is None: + return [] if definition.pair_fluents else None + return [list(triple) for triple in given] + + def _argument(definition: RuntimeDefinition, x: Any) -> Any: if isinstance(x, Mapping): return Mean(str(x["mean"])) @@ -753,7 +1022,11 @@ def _runtime_comparison( def _brute_event_kind( - brute: BruteForce, event: BruteEvent, state: BruteState, numeric: BruteNumeric | None = None + brute: BruteForce, + event: BruteEvent, + state: BruteState, + numeric: BruteNumeric | None = None, + pairs: BrutePairs | None = None, ) -> str | None: """``illegal`` when the brute-force evaluator finds the event malformed or not legal.""" if event.event_type not in brute.performable(): @@ -763,7 +1036,14 @@ def _brute_event_kind( return "illegal" if any(label not in brute.entities for label in event.binding.values()): return "illegal" - return None if brute.legal(event, state, numeric) else "illegal" + return None if brute.legal(event, state, numeric, pairs) else "illegal" + + +def _brute_initial_pairs(brute: BruteForce, fixture: Mapping[str, Any]) -> BrutePairs: + given = fixture.get("initial_pairs") + if given is None: + return brute.empty_pairs() + return brute.pairs_from_record(given) def _brute_initial_numeric(brute: BruteForce, fixture: Mapping[str, Any]) -> BruteNumeric | None: @@ -784,10 +1064,11 @@ def check_fixture_brute(fixture: Mapping[str, Any]) -> FixtureReport: brute = BruteForce(fixture["definition"], fixture["entities"], fixture.get("places", ())) state: BruteState = {e: frozenset(fixture["initial"].get(e, ())) for e in brute.entity_labels} numeric = _brute_initial_numeric(brute, fixture) + pairs = _brute_initial_pairs(brute, fixture) numerics: list[BruteNumeric | None] = [numeric] error = fixture["error"] for k, step in enumerate(fixture["steps"], start=1): - legal = brute.legal_record(state, numeric) + legal = brute.legal_record(state, numeric, pairs) if legal != {label: list(b) for label, b in step["legal"].items()}: raise _fail( name, @@ -795,7 +1076,9 @@ def check_fixture_brute(fixture: Mapping[str, Any]) -> FixtureReport: f"{step['legal']}", ) events = [BruteEvent(e["event_type"], dict(e["binding"])) for e in step.get("events", [])] - kinds = [k for k in (_brute_event_kind(brute, e, state, numeric) for e in events) if k] + kinds = [ + k for k in (_brute_event_kind(brute, e, state, numeric, pairs) for e in events) if k + ] if not kinds and any(brute.interfere(a, b) for a, b in itertools.combinations(events, 2)): kinds.append("interference") found = kinds[0] if kinds else None @@ -809,20 +1092,31 @@ def check_fixture_brute(fixture: Mapping[str, Any]) -> FixtureReport: return FixtureReport(name, len(fixture["steps"]), error["kind"]) if found is not None: raise _fail(name, f"step {k}: the brute-force evaluator finds the step {found}") - state, numeric, _ = brute.step(state, numeric, events) + state, numeric, _ = brute.step(state, numeric, events, pairs) + pairs = brute.pairs_after if brute.state_record(state) != {e: list(v) for e, v in step["state"].items()}: raise _fail( name, f"step {k}: the brute-force state is {brute.state_record(state)}, but the " f"fixture expects {step['state']}", ) - if brute.derived_record(state, numeric) != {e: list(v) for e, v in step["derived"].items()}: + if brute.derived_record(state, numeric, pairs) != { + e: list(v) for e, v in step["derived"].items() + }: raise _fail( name, f"step {k}: the brute-force derived fluents are " - f"{brute.derived_record(state, numeric)}, " + f"{brute.derived_record(state, numeric, pairs)}, " f"but the fixture expects {step['derived']}", ) + if brute.pair_fluents or "pairs" in step: + expected_pairs = [list(t) for t in step.get("pairs", [])] + if brute.pairs_record(pairs) != expected_pairs: + raise _fail( + name, + f"step {k}: the brute-force pairs are {brute.pairs_record(pairs)}, but the " + f"fixture expects {expected_pairs}", + ) if brute.numeric_fluents or "numeric" in step: expected = { e: {f: float(x) for f, x in v.items()} for e, v in step.get("numeric", {}).items() @@ -909,13 +1203,15 @@ def generate_fixture( error: str | None = None, initial_numeric: Mapping[str, Mapping[str, float]] | None = None, places: Sequence[Mapping[str, Any]] | None = None, + initial_pairs: Sequence[Sequence[str]] | None = None, ) -> dict[str, Any]: """A fixture whose expected values come from the brute-force evaluator. Each step draws a random set of up to ``max_events`` non-interfering legal events. With ``error``, the last step instead holds an illegal event (``illegal``) or two interfering legal events (``interference``), found among the events of that state; the search fails with :class:`WorldError` when the state offers none. ``places`` (stage c1) is the places table - of a world with places.""" + of a world with places; ``initial_pairs`` (stage c2a) the pairs at ``TIME.1`` of a world + with pair fluents.""" brute = BruteForce(record, entities, places or ()) rng = np.random.default_rng(seed) state: BruteState = {e: frozenset(initial.get(e, ())) for e in brute.entity_labels} @@ -926,29 +1222,35 @@ def generate_fixture( ) if numeric is None and brute.numeric_fluents: raise WorldError("the world has numeric fluents, so the fixture needs their initial values") + pairs = brute.pairs_from_record(initial_pairs or []) + if initial_pairs is None and brute.pair_fluents: + raise WorldError("the world has pair fluents, so the fixture needs its initial pairs") out_steps: list[dict[str, Any]] = [] error_entry = None for k in range(1, steps + 1): - legal_record = brute.legal_record(state, numeric) + legal_record = brute.legal_record(state, numeric, pairs) candidates = [ BruteEvent(label, b) for label, bindings in legal_record.items() for b in bindings ] if error is not None and k == steps: - events = _error_events(brute, candidates, state, rng, error, numeric) + events = _error_events(brute, candidates, state, rng, error, numeric, pairs) out_steps.append({"events": [_event_record(e) for e in events], "legal": legal_record}) error_entry = {"step": k, "kind": error} break size = int(rng.integers(0, max_events + 1)) events = _independent_set(brute, candidates, rng, size) - state, numeric, _ = brute.step(state, numeric, events) + state, numeric, _ = brute.step(state, numeric, events, pairs) + pairs = brute.pairs_after entry = { "events": [_event_record(e) for e in events], "legal": legal_record, - "derived": brute.derived_record(state, numeric), + "derived": brute.derived_record(state, numeric, pairs), "state": brute.state_record(state), } if brute.numeric_fluents: entry["numeric"] = brute.numeric_record(numeric) + if brute.pair_fluents: + entry["pairs"] = brute.pairs_record(pairs) out_steps.append(entry) return fixture_record( name, @@ -960,6 +1262,7 @@ def generate_fixture( error_entry, initial_numeric, places=places, + initial_pairs=initial_pairs, ) @@ -970,13 +1273,14 @@ def _error_events( rng: np.random.Generator, kind: str, numeric: BruteNumeric | None = None, + pairs: BrutePairs | None = None, ) -> list[BruteEvent]: if kind == "illegal": illegal = [ BruteEvent(label, b) for label in brute.performable() for b in brute.bindings(label) - if not brute.legal(BruteEvent(label, b), state, numeric) + if not brute.legal(BruteEvent(label, b), state, numeric, pairs) ] if not illegal: raise WorldError("every event is legal in the state; no illegal event to draw") @@ -1016,6 +1320,17 @@ def fixture_places(definition: Definition) -> list[dict[str, Any]] | None: ) +def fixture_initial_pairs(definition: Definition) -> list[list[str]] | None: + """A built definition's initial pairs as a fixture gives them (the rows of + ``initial_pairs.csv`` as triples, through their JSON form), or None without pair fluents + (stage c2a).""" + if not len(definition.pair_fluents): + return None + frame = initial_pairs_frame(definition.pair_fluents, definition.taxonomy.instances.labels) + rows = json.loads(to_json(frame.to_dicts(), indent=1, sort_keys=False)) + return [[row["pair_fluent"], row["first"], row["second"]] for row in rows] + + def fixture_initial_numeric(definition: Definition) -> dict[str, dict[str, float]] | None: """A built definition's initial numeric values as a fixture gives them, or None without numeric fluents.""" @@ -1034,6 +1349,7 @@ def world_fixtures( record, entities, initial = fixture_inputs(definition) initial_numeric = fixture_initial_numeric(definition) places = fixture_places(definition) + initial_pairs = fixture_initial_pairs(definition) fixtures = [] for seed in range(1, count + 1): fixtures.append( @@ -1048,6 +1364,7 @@ def world_fixtures( steps, initial_numeric=initial_numeric, places=places, + initial_pairs=initial_pairs, ) ) for kind, text in ( @@ -1067,6 +1384,7 @@ def world_fixtures( error=kind, initial_numeric=initial_numeric, places=places, + initial_pairs=initial_pairs, ) ) return fixtures @@ -1095,6 +1413,7 @@ def write_world_fixtures( "BruteEvent", "BruteForce", "BruteNumeric", + "BrutePairs", "BruteState", "FixtureError", "FixtureReport", @@ -1102,6 +1421,7 @@ def write_world_fixtures( "check_fixture_brute", "check_fixtures", "fixture_initial_numeric", + "fixture_initial_pairs", "fixture_inputs", "fixture_paths", "fixture_record", diff --git a/python/semantic_world/world/generate.py b/python/semantic_world/world/generate.py index 27e81f4..a16fc30 100644 --- a/python/semantic_world/world/generate.py +++ b/python/semantic_world/world/generate.py @@ -40,6 +40,7 @@ ) from semantic_world.world.fluents import Fluents, derived_initial_values, generate_fluents from semantic_world.world.numeric import NumericFluents, generate_numeric_fluents +from semantic_world.world.pair_fluents import PairFluents, generate_pair_fluents from semantic_world.world.places import ( PLACE_CAPACITIES_FILE, PLACES_DIR, @@ -89,6 +90,12 @@ class WorldResult: def rule_set_id(self) -> str: return self.definition.rule_set_id + @property + def pair_fluents(self) -> PairFluents: + """The pair fluents between entities and their initial pairs (stage c2a), from the + event file; none without one.""" + return self.definition.pair_fluents + def derived_frames(self) -> dict[str, pl.DataFrame]: """Every table of ``derived/``, keyed by file name. With places, ``capacities.csv`` holds the entities' ``CAN.PLACEEVENTTYPE`` columns, ``place_capacities.csv`` the @@ -162,6 +169,13 @@ def define(config: Config) -> WorldResult: streams = WorldStreams(config.seed) statics = build_statics(taxonomy, config, streams, event_file) places = generate_places(config, taxonomy, streams) + pair_fluents = generate_pair_fluents( + len(taxonomy.instances), + streams, + explicit=event_file.pair_fluents if event_file is not None else None, + explicit_source=event_file.source if event_file is not None else None, + ) + spatial = event_file.spatial if event_file is not None else None numeric = generate_numeric_fluents( config, taxonomy, @@ -179,12 +193,14 @@ def define(config: Config) -> WorldResult: ) derived_initial = derived_initial_values(fluents, taxonomy, numeric) event_types = generate_event_types( - statics, fluents, config, streams, event_file, derived_initial, numeric + statics, fluents, config, streams, event_file, derived_initial, numeric, pair_fluents ) initial_present = _initial_present(fluents, derived_initial) features = tuple(f.label for f in taxonomy.features.features) - _check(event_types, fluents, initial_present, features, numeric) - definition = build_definition(statics, fluents, event_types, numeric, places) + _check(event_types, fluents, initial_present, features, numeric, pair_fluents) + definition = build_definition( + statics, fluents, event_types, numeric, places, pair_fluents, spatial + ) fixed = set() if event_file is not None: fixed = { @@ -211,8 +227,10 @@ def define(config: Config) -> WorldResult: event_types = redraw_preconditions( event_types, never, fluents, config, parts, derived_initial, numeric, numeric_parts ) - _check(event_types, fluents, initial_present, features, numeric) - definition = build_definition(statics, fluents, event_types, numeric, places) + _check(event_types, fluents, initial_present, features, numeric, pair_fluents) + definition = build_definition( + statics, fluents, event_types, numeric, places, pair_fluents, spatial + ) episodes = statistics_episodes(definition, statics, config, redraws) report = check_definition(definition) stats = world_stats(fluents, event_types, episodes, statics, numeric, places) @@ -243,9 +261,10 @@ def _check( initial_present: set, features: tuple[str, ...], numeric: NumericFluents | None = None, + pair_fluents: PairFluents | None = None, ) -> None: try: - check_dynamics(event_types, fluents, initial_present, features, numeric) + check_dynamics(event_types, fluents, initial_present, features, numeric, pair_fluents) except ValueError as error: raise WorldError(f"the event types break a rule of the dynamics: {error}") from None diff --git a/python/semantic_world/world/history.py b/python/semantic_world/world/history.py index 528fb8a..555a5ed 100644 --- a/python/semantic_world/world/history.py +++ b/python/semantic_world/world/history.py @@ -10,12 +10,19 @@ drift, and each step records the drift's changes apart under ``drift`` (``"cause": "drift"``), so that every numeric change of a step is recorded with its cause; a history's ``initial_numeric`` holds each participant's numeric values at ``TIME.1``. A world without numeric fluents writes -none of these keys, so its histories are as before. +none of these keys, so its histories are as before. Since stage c2a a world with pair fluents +(``docs/specs/WORLD_PHASE_C.md``, "Histories") writes ``initial_pairs`` after ``initial_numeric`` +(every ordered pair that holds among the participants at ``TIME.1``, as ``[pair fluent, first, +second]`` triples, both directions of a symmetric pair) and records a pair change as ``{"pair", +"first", "second", "to"}``, a symmetric pair once, in the direction the effect names; a world +without pair fluents writes neither. ``replay`` runs a history's events through the runtime from its initial state and checks that every step is legal and free of interference, that the recorded Boolean changes are exactly the -base fluents that changed, and that every recorded numeric value is reproduced exactly, by the -same operations in the same order (``clamp(clamp(v + e) + d)``, in float64). +base fluents that changed, that every recorded numeric value is reproduced exactly, by the +same operations in the same order (``clamp(clamp(v + e) + d)``, in float64), and that the +recorded pair changes are exactly the pairs the events changed, both directions of a symmetric +pair written from one recorded change. """ from __future__ import annotations @@ -58,6 +65,10 @@ class Change: def numeric(self) -> bool: return self.from_ is not None + @property + def pair(self) -> bool: + return False + def to_json(self) -> dict[str, Any]: if self.numeric: return { @@ -82,6 +93,51 @@ def from_json(cls, data: Mapping[str, Any]) -> Change: return cls(str(data["entity"]), str(data["fluent"]), bool(data["to"])) +@dataclass(frozen=True) +class PairChange: + """One pair an event added or deleted (stage c2a): the pair fluent, its two arguments in + the direction the effect names, and the new value. A symmetric pair is recorded once; + replay writes both directions.""" + + pair_fluent: str + first: str + second: str + to: bool + + @property + def numeric(self) -> bool: + return False + + @property + def pair(self) -> bool: + return True + + def to_json(self) -> dict[str, Any]: + return { + "pair": self.pair_fluent, + "first": self.first, + "second": self.second, + "to": bool(self.to), + } + + @classmethod + def from_json(cls, data: Mapping[str, Any]) -> PairChange: + return cls(str(data["pair"]), str(data["first"]), str(data["second"]), bool(data["to"])) + + +AnyChange = Change | PairChange + + +def change_from_json(data: Mapping[str, Any]) -> AnyChange: + if "pair" in data: + return PairChange.from_json(data) + return Change.from_json(data) + + +PairTriple = tuple[str, str, str] +"""``(pair fluent, first, second)``: one ordered pair that holds, by labels.""" + + @dataclass(frozen=True) class HistoryEvent: label: str @@ -89,7 +145,7 @@ class HistoryEvent: type: str agent: str patient: str | None = None - changes: tuple[Change, ...] = () + changes: tuple[AnyChange, ...] = () @property def binding(self) -> dict[str, str]: @@ -112,7 +168,7 @@ def from_json(cls, data: Mapping[str, Any]) -> HistoryEvent: str(data["type"]), str(data["agent"]), None if data.get("patient") is None else str(data["patient"]), - tuple(Change.from_json(c) for c in data.get("changes", [])), + tuple(change_from_json(c) for c in data.get("changes", [])), ) @@ -165,11 +221,26 @@ class History: initial_numeric: dict[str, dict[str, float]] = field(default_factory=dict) """For each participant, its numeric fluents at ``TIME.1`` (stage b2); empty in a world without numeric fluents, and then not written.""" + initial_pairs: tuple[PairTriple, ...] | None = None + """Every ordered pair that holds among the participants at ``TIME.1`` (stage c2a), as + ``(pair fluent, first, second)``; None in a world without pair fluents, and then not + written (an empty tuple is written, as ``[]``).""" @property def events(self) -> tuple[HistoryEvent, ...]: return tuple(e for step in self.steps for e in step.events) + @property + def pair_changes(self) -> tuple[PairChange, ...]: + """Every pair change of the history, in time order.""" + return tuple( + c + for step in self.steps + for e in step.events + for c in e.changes + if isinstance(c, PairChange) + ) + @property def numeric_changes(self) -> tuple[Change, ...]: """Every numeric change of the history, the effects' and the drift's, in time order.""" @@ -194,6 +265,8 @@ def to_json(self) -> dict[str, Any]: p: {f: float(v) for f, v in self.initial_numeric[p].items()} for p in self.participants } + if self.initial_pairs is not None: + data["initial_pairs"] = [list(triple) for triple in self.initial_pairs] data["steps"] = [step.to_json() for step in self.steps] data["final"] = self.final data["quiescent"] = self.quiescent @@ -215,6 +288,9 @@ def from_json(cls, data: Mapping[str, Any]) -> History: str(p): {str(f): float(v) for f, v in values.items()} for p, values in (data.get("initial_numeric") or {}).items() }, + None + if data.get("initial_pairs") is None + else tuple((str(a), str(b), str(c)) for a, b, c in data["initial_pairs"]), ) @@ -257,8 +333,22 @@ def initial_state_of(definition: RuntimeDefinition, history: History) -> State: row = definition.entity_index(participant) for fluent, value in values.items(): numeric[row, definition.numeric_index(fluent)] = float(value) - state = State.from_true(definition, true, {}) - return State(state.values, numeric) + # The pairs (stage c2a): a history of a world with pair fluents lists every pair that holds + # among its participants, and an episode keeps no other pair (WM.38), so the history's + # initial pairs are the whole pair part of the state. + pairs: list = [] + if definition.pair_fluents: + if history.initial_pairs is None: + raise HistoryError( + f"{history.label} lists no initial pairs, but the definition has pair fluents" + ) + pairs = [list(triple) for triple in history.initial_pairs] + elif history.initial_pairs: + raise HistoryError( + f"{history.label} lists initial pairs, but the definition has no pair fluents" + ) + state = State.from_true(definition, true, {}, pairs) + return State(state.values, numeric, state.pairs) def changes_of( @@ -304,6 +394,28 @@ def numeric_changes_of( return found +def pair_changes_of( + definition: RuntimeDefinition, before: State, after: State, event: Event +) -> list[PairChange]: + """The pairs the event's pair effects changed between two states, each in the direction the + effect names (a symmetric pair once), in effect order.""" + et = definition.event_type(event.event_type) + found: list[PairChange] = [] + labels = definition.entity_labels + for effect in et.effects: + if not effect.pair: + continue + assert effect.second is not None + k = definition.pair_index(effect.fluent) + i = event.binding[et.roles.index(effect.role)] + j = event.binding[et.roles.index(effect.second)] + if before.pairs[k][i, j] != after.pairs[k][i, j]: + found.append( + PairChange(effect.fluent, labels[i], labels[j], bool(after.pairs[k][i, j])) + ) + return found + + def replay(definition: RuntimeDefinition, history: History) -> list[State]: """Run the history's events through ``apply`` from its initial state. Returns the state at every time point. Raises :class:`HistoryError` when a step does not apply, or when an @@ -323,7 +435,7 @@ def replay(definition: RuntimeDefinition, history: History) -> list[State]: except WorldError as error: raise HistoryError(f"{history.label}, step {step.step}: {error}") from None after = result.state - recorded = [c for e in step.events for c in e.changes if not c.numeric] + recorded = [c for e in step.events for c in e.changes if not c.numeric and not c.pair] found = changes_of( definition, state, after, sorted(set(i for e in events for i in e.binding)) ) @@ -337,13 +449,25 @@ def replay(definition: RuntimeDefinition, history: History) -> list[State]: written = { (definition.entity_labels[event.binding[et.roles.index(e.role)]], e.fluent) for e in et.effects + if not e.pair } for change in recorded_event.changes: + if change.pair: + continue if (change.entity, change.fluent) not in written: raise HistoryError( f"{history.label}, step {step.step}: {recorded_event.label} records a " f"change to {change.fluent} of {change.entity}, which it does not write" ) + # The pair changes of the event (stage c2a): exactly the pairs its pair effects + # changed, each in the effect's direction. + expected_pairs = pair_changes_of(definition, state, after, event) + got_pairs = [c for c in recorded_event.changes if c.pair] + if sorted(got_pairs, key=_pair_key) != sorted(expected_pairs, key=_pair_key): + raise HistoryError( + f"{history.label}, step {step.step}: {recorded_event.label} records the " + f"pair changes {got_pairs}, but its effects make {expected_pairs}" + ) # The numeric changes of the event: its numeric effects, judged after the effect # and before the drift, reproduced exactly. expected = numeric_changes_of( @@ -382,6 +506,10 @@ def _change_key(change: Change) -> tuple[str, str, bool]: return (change.entity, change.fluent, bool(change.to)) +def _pair_key(change: PairChange) -> tuple[str, str, str, bool]: + return (change.pair_fluent, change.first, change.second, bool(change.to)) + + def _numeric_key(change: Change) -> tuple[str, str, float, float, str]: return ( change.entity, @@ -399,8 +527,11 @@ def _numeric_key(change: Change) -> tuple[str, str, float, float, str]: "History", "HistoryError", "HistoryEvent", + "PairChange", "Step", + "change_from_json", "changes_of", + "pair_changes_of", "initial_state_of", "numeric_changes_of", "read_histories", diff --git a/python/semantic_world/world/pair_fluents.py b/python/semantic_world/world/pair_fluents.py new file mode 100644 index 0000000..8f9aa2b --- /dev/null +++ b/python/semantic_world/world/pair_fluents.py @@ -0,0 +1,395 @@ +"""Pair fluents between entities (stage c2a of ``docs/specs/WORLD_PHASE_C.md``, "Pair fluents"). + +A pair fluent ``PAIRFL.`` is a Boolean fluent of two arguments, stored as the set of ordered +pairs for which it holds at a time point. Its profile is its symmetry (``symmetric``: ``(x, y)`` +holds exactly when ``(y, x)`` holds; ``asymmetric``: never both; ``general``: no constraint) +and whether it is ``functional`` (at most one pair in each row, and for a symmetric functional +pair fluent at most one partner in each row and each column). Every pair fluent is irreflexive. +A pair fluent between entities is never total (WM.38); the total functional profile is the +location fluents' alone, toward places (stage c3). + +In this stage pair fluents come from an event file alone (``pair_fluents`` in +``event_file.py``): the generator draws nothing from ``world:pair_fluents``, which exists so +that the order of the streams is fixed for stage c2b. The initial pairs are drawn from +``world:pair_initial`` at each pair fluent's initial rate, respecting the profile ("Initial pairs +between entities"): a symmetric pair fluent draws each unordered pair at the rate and writes both +directions; an asymmetric one draws each unordered pair at the rate and then one direction; a +general one draws each ordered pair at the rate; a functional one gives each entity, at the rate, +one partner drawn uniformly among the entities that still keep the profile. The pairs are written +to ``initial_pairs.csv``, one ordered pair per row (both directions of a symmetric pair). +""" + +from __future__ import annotations + +from collections.abc import Mapping, Sequence +from dataclasses import dataclass +from typing import Any + +import numpy as np +import polars as pl + +from semantic_world.taxonomy.config import ConfigError +from semantic_world.world.errors import WorldError + +PAIR_PREFIX = "PAIRFL" +PAIR_FLUENT_KIND = "pair_fluent" +"""The symbol kind of a pair fluent in ``definition.json``.""" +SYMMETRIES = ("symmetric", "asymmetric", "general") +INITIAL_PAIRS_FILE = "initial_pairs.csv" +INITIAL_PAIRS_COLUMNS = ("pair_fluent", "first", "second") +"""The columns of ``initial_pairs.csv``: the pair fluent's label, and the labels of the first +and the second argument of one ordered pair that holds at ``TIME.1``.""" + + +def pair_label(index: int) -> str: + return f"{PAIR_PREFIX}.{index}" + + +def is_pair_label(label: str) -> bool: + prefix, _, number = label.partition(".") + return prefix == PAIR_PREFIX and number.isdigit() + + +@dataclass(frozen=True) +class PairFluent: + """One pair fluent between entities and its profile.""" + + label: str + index: int + symmetry: str + functional: bool + initial_rate: float + total: bool = False + """Always False for a pair fluent between entities; a location fluent (stage c3) is total.""" + + def __post_init__(self) -> None: + if self.symmetry not in SYMMETRIES: + raise WorldError(f"{self.label}: unknown symmetry {self.symmetry!r}") + if not 0.0 <= self.initial_rate <= 1.0: + raise WorldError(f"{self.label}: the initial rate is not between 0 and 1") + + @property + def symmetric(self) -> bool: + return self.symmetry == "symmetric" + + @property + def asymmetric(self) -> bool: + return self.symmetry == "asymmetric" + + def symbol_record(self) -> dict[str, Any]: + """The symbol of ``definition.json``: a fluent of arity 2 with its profile and its + initial rate, all hashed into the rule-set identity.""" + return { + "label": self.label, + "kind": PAIR_FLUENT_KIND, + "derived": False, + "fluent": True, + "arity": 2, + "symmetry": self.symmetry, + "functional": self.functional, + "total": self.total, + "initial_rate": float(self.initial_rate), + } + + @classmethod + def from_record(cls, data: Mapping[str, Any]) -> PairFluent: + label = str(data["label"]) + prefix, _, number = label.partition(".") + if prefix != PAIR_PREFIX or not number.isdigit(): + raise WorldError(f"{label} is not a pair fluent label") + return cls( + label, + int(number), + str(data["symmetry"]), + bool(data["functional"]), + float(data["initial_rate"]), + bool(data.get("total", False)), + ) + + +@dataclass(frozen=True) +class PairFluents: + """The pair fluents of a world and their initial pairs, one Boolean matrix per pair fluent, + rows the entities and columns the targets (the entities in this stage, the places for a + location fluent in stage c3).""" + + fluents: tuple[PairFluent, ...] + initial_pairs: tuple[np.ndarray, ...] + + def __post_init__(self) -> None: + if len(self.initial_pairs) != len(self.fluents): + raise WorldError("every pair fluent needs its initial pairs") + object.__setattr__(self, "_by_label", {f.label: f for f in self.fluents}) + object.__setattr__(self, "_index", {f.label: i for i, f in enumerate(self.fluents)}) + for matrix in self.initial_pairs: + matrix.setflags(write=False) + + def __len__(self) -> int: + return len(self.fluents) + + def __iter__(self): + return iter(self.fluents) + + def __getitem__(self, label: str) -> PairFluent: + try: + return self._by_label[label] # type: ignore[attr-defined] + except KeyError: + raise KeyError(f"unknown pair fluent {label!r}") from None + + def __contains__(self, label: object) -> bool: + return label in self._by_label # type: ignore[attr-defined] + + def index(self, label: str) -> int: + return self._index[label] # type: ignore[attr-defined] + + @property + def labels(self) -> tuple[str, ...]: + return tuple(f.label for f in self.fluents) + + +EMPTY = PairFluents((), ()) + + +def empty_pair_fluents() -> PairFluents: + return EMPTY + + +# --------------------------------------------------------------------------------------------- +# Profiles +# --------------------------------------------------------------------------------------------- + + +def profile_violations(fluent: PairFluent, matrix: np.ndarray) -> list[str]: + """Why a pair matrix breaks the pair fluent's profile: each violation as a sentence, none + when the profile holds. A pair fluent between entities is irreflexive; symmetric: every + pair with its reverse; asymmetric: no pair with its reverse; functional: at most one pair + in each row (and in each column of a symmetric one, which symmetry already gives).""" + found: list[str] = [] + m = np.asarray(matrix, dtype=bool) + if m.ndim != 2: + raise WorldError(f"the pairs of {fluent.label} are a 2-D matrix, got {m.shape}") + if m.shape[0] == m.shape[1]: + diagonal = np.flatnonzero(np.diagonal(m)) + if diagonal.size: + found.append(f"{fluent.label} holds between entity {int(diagonal[0])} and itself") + if fluent.symmetric: + rows, columns = np.nonzero(m & ~m.T) + if rows.size: + found.append( + f"{fluent.label} is symmetric, but ({int(rows[0])}, {int(columns[0])}) " + f"holds without ({int(columns[0])}, {int(rows[0])})" + ) + if fluent.asymmetric: + rows, columns = np.nonzero(m & m.T) + if rows.size: + found.append( + f"{fluent.label} is asymmetric, but ({int(rows[0])}, {int(columns[0])}) " + f"and ({int(columns[0])}, {int(rows[0])}) both hold" + ) + if fluent.functional: + counts = m.sum(axis=1) + over = np.flatnonzero(counts > 1) + if over.size: + found.append( + f"{fluent.label} is functional, but the row of entity {int(over[0])} holds " + f"{int(counts[over[0]])} pairs" + ) + if fluent.total: + counts = m.sum(axis=1) + empty = np.flatnonzero(counts == 0) + if empty.size: + found.append( + f"{fluent.label} is total, but the row of entity {int(empty[0])} holds no pair" + ) + return found + + +# --------------------------------------------------------------------------------------------- +# Initial pairs +# --------------------------------------------------------------------------------------------- + + +def draw_initial_pairs(fluent: PairFluent, n: int, rng: np.random.Generator) -> np.ndarray: + """The initial pairs of a pair fluent between ``n`` entities, drawn from ``rng`` at the + fluent's initial rate by the procedure of "Initial pairs between entities", as a Boolean + matrix. The draws are made in a fixed order: for a functional pair fluent one draw against + the rate per entity, in entity order, then the partner uniformly among the entities that + keep the profile (none when no entity does); otherwise one draw per unordered pair (``i < + j``, row by row) for a symmetric or an asymmetric pair fluent, with one more draw for the + direction of an asymmetric pair, and one draw per ordered pair (row by row) for a general + one. A rate of 0 draws as the others do and selects nothing.""" + matrix = np.zeros((n, n), dtype=bool) + rate = float(fluent.initial_rate) + if fluent.functional: + for x in range(n): + if rng.random() >= rate: + continue + candidates = _partners_keeping_profile(fluent, matrix, x) + if not candidates: + continue + y = candidates[int(rng.integers(len(candidates)))] + matrix[x, y] = True + if fluent.symmetric: + matrix[y, x] = True + return matrix + if fluent.symmetry == "general": + for x in range(n): + for y in range(n): + if x != y and rng.random() < rate: + matrix[x, y] = True + return matrix + for x in range(n): + for y in range(x + 1, n): + if rng.random() < rate: + if fluent.symmetric: + matrix[x, y] = matrix[y, x] = True + elif rng.integers(2) == 0: + matrix[x, y] = True + else: + matrix[y, x] = True + return matrix + + +def _partners_keeping_profile(fluent: PairFluent, matrix: np.ndarray, x: int) -> list[int]: + """The entities ``y`` such that adding ``(x, y)`` (and ``(y, x)`` for a symmetric pair + fluent) keeps the functional profile: the row of ``x`` is empty, ``y`` is not ``x``, for an + asymmetric pair fluent ``(y, x)`` does not hold, and for a symmetric one the row of ``y`` is + empty too.""" + n = matrix.shape[0] + if matrix[x].any(): + return [] + found = [] + for y in range(n): + if y == x: + continue + if fluent.asymmetric and matrix[y, x]: + continue + if fluent.symmetric and matrix[y].any(): + continue + found.append(y) + return found + + +def generate_pair_fluents( + entity_count: int, + streams, + explicit: Mapping[str, Mapping[str, Any]] | None = None, + explicit_source: str | None = None, +) -> PairFluents: + """The pair fluents of a world and their initial pairs. In stage c2a the pair fluents are + those of the event file (``explicit``: ``{label: {symmetry, functional, initial_rate}}``), + labeled ``PAIRFL.1`` to ``PAIRFL.k`` without a gap; nothing is drawn from + ``world:pair_fluents``. The initial pairs come from ``world:pair_initial``, fluent by fluent + in label order. Without pair fluents nothing is drawn at all.""" + if not explicit: + return EMPTY + source = explicit_source or "" + fluents: list[PairFluent] = [] + expected = [pair_label(i) for i in range(1, len(explicit) + 1)] + labels = sorted(explicit, key=lambda label: int(label.split(".")[1])) + if labels != expected: + raise ConfigError( + source, + "pair_fluents", + f"the pair fluents are labeled {', '.join(labels)}; they must be " + f"{expected[0]} to {expected[-1]} without a gap", + ) + for label in expected: + setting = explicit[label] + fluents.append( + PairFluent( + label, + int(label.split(".")[1]), + str(setting["symmetry"]), + bool(setting["functional"]), + float(setting["initial_rate"]), + ) + ) + rng = streams.pair_initial + initial = tuple(draw_initial_pairs(fluent, entity_count, rng) for fluent in fluents) + for fluent, matrix in zip(fluents, initial, strict=True): + violations = profile_violations(fluent, matrix) + if violations: + raise WorldError(f"the initial pairs break a profile: {violations[0]}") + return PairFluents(tuple(fluents), initial) + + +# --------------------------------------------------------------------------------------------- +# initial_pairs.csv +# --------------------------------------------------------------------------------------------- + + +def pair_rows( + fluents: Sequence[PairFluent], matrices: Sequence[np.ndarray], labels: Sequence[str] +) -> list[tuple[str, str, str]]: + """Every ordered pair that holds, as ``(pair fluent, first, second)`` label triples, by pair + fluent in label order, then by first and second in entity order; both directions of a + symmetric pair.""" + rows: list[tuple[str, str, str]] = [] + for fluent, matrix in zip(fluents, matrices, strict=True): + for x, y in zip(*np.nonzero(np.asarray(matrix, dtype=bool)), strict=True): + rows.append((fluent.label, labels[int(x)], labels[int(y)])) + return rows + + +def initial_pairs_frame(pair_fluents: PairFluents, labels: Sequence[str]) -> pl.DataFrame: + """``initial_pairs.csv``: one row per ordered pair that holds at ``TIME.1``.""" + rows = pair_rows(pair_fluents.fluents, pair_fluents.initial_pairs, labels) + return pl.DataFrame( + { + "pair_fluent": [r[0] for r in rows], + "first": [r[1] for r in rows], + "second": [r[2] for r in rows], + }, + schema={name: pl.Utf8 for name in INITIAL_PAIRS_COLUMNS}, + ) + + +def matrices_from_rows( + rows: Sequence[Mapping[str, Any]] | Sequence[Sequence[str]], + fluents: Sequence[PairFluent], + entity_index: Mapping[str, int], + what: str = "the initial pairs", +) -> tuple[np.ndarray, ...]: + """The pair matrices from rows of ``(pair fluent, first, second)``, given as mappings with + those keys (the rows of ``initial_pairs.csv``) or as triples (a history's or a fixture's + ``initial_pairs``). An unknown pair fluent or entity, or a pair of an entity with itself, + is a :class:`WorldError`.""" + n = len(entity_index) + index = {f.label: i for i, f in enumerate(fluents)} + matrices = [np.zeros((n, n), dtype=bool) for _ in fluents] + for row in rows: + if isinstance(row, Mapping): + fluent, first, second = str(row["pair_fluent"]), str(row["first"]), str(row["second"]) + else: + fluent, first, second = (str(v) for v in row) + if fluent not in index: + raise WorldError(f"{what} name {fluent}, which is not a pair fluent of the definition") + for label in (first, second): + if label not in entity_index: + raise WorldError(f"{what} name {label}, which is not an entity of the definition") + if first == second: + raise WorldError(f"{what} hold {fluent} between {first} and itself") + matrices[index[fluent]][entity_index[first], entity_index[second]] = True + return tuple(matrices) + + +__all__ = [ + "EMPTY", + "INITIAL_PAIRS_COLUMNS", + "INITIAL_PAIRS_FILE", + "PAIR_FLUENT_KIND", + "PAIR_PREFIX", + "SYMMETRIES", + "PairFluent", + "PairFluents", + "draw_initial_pairs", + "empty_pair_fluents", + "generate_pair_fluents", + "initial_pairs_frame", + "is_pair_label", + "matrices_from_rows", + "pair_label", + "pair_rows", + "profile_violations", +] diff --git a/python/semantic_world/world/runtime.py b/python/semantic_world/world/runtime.py index a0a720a..1ad23b7 100644 --- a/python/semantic_world/world/runtime.py +++ b/python/semantic_world/world/runtime.py @@ -24,6 +24,17 @@ the values after the effects and before the drift, which histories record. The comparisons of "Phase (b): the world" (``greater``, ``above``, ``below``, ``increase``, ``decrease``) are judged on demand by direct arithmetic on the values, for entities or category means. + +Since stage c2a (``docs/specs/WORLD_PHASE_C.md``, "The runtime") a state also holds, for each +pair fluent, the set of its pairs among the entities, as a Boolean matrix whose rows are the +entities and whose columns are the fluent's targets (entities; places for a location fluent in +stage c3). ``derive`` computes the pair literals ``SOME`` and ``SOMEOF`` before the matrices; +``legal`` judges the pair literals of a precondition over the binding; ``apply`` writes the pairs +its effects add or delete, both directions of a symmetric pair, and checks every profile after +the step (:class:`ProfileViolationError`, which a correct definition never raises). An event reads +the pairs, rows, and columns its precondition and conditions read, through the cones of derived +fluents, and writes the pairs its effects write; two events interfere when one writes a pair the +other writes or reads, a row or a column counting as every pair in it. """ from __future__ import annotations @@ -34,10 +45,22 @@ import numpy as np from semantic_world.world.definition import EventTypeRecord, RuntimeDefinition -from semantic_world.world.dynamics import Effect -from semantic_world.world.errors import IllegalEventError, InterferenceError, WorldError +from semantic_world.world.dynamics import ( + PAIR_KIND, + PAIR_SOME, + PAIR_SOMEOF, + Effect, + PairLiteral, +) +from semantic_world.world.errors import ( + IllegalEventError, + InterferenceError, + ProfileViolationError, + WorldError, +) from semantic_world.world.matrices import AgreementReport, check_agreement from semantic_world.world.numeric import clamp +from semantic_world.world.pair_fluents import matrices_from_rows, pair_rows, profile_violations from semantic_world.world.places import EVERYPLACE Binding = tuple[int, ...] @@ -58,12 +81,15 @@ @dataclass(frozen=True) class State: """The base fluents of every entity at one time point: ``values``, shape ``(entities, base - fluents)`` with 0 and 1, and ``numeric``, shape ``(entities, numeric fluents)`` in float64 - (stage b2; a world without numeric fluents has an empty second axis). The arrays are - read-only; ``apply`` returns a new state.""" + fluents)`` with 0 and 1, ``numeric``, shape ``(entities, numeric fluents)`` in float64 + (stage b2; a world without numeric fluents has an empty second axis), and ``pairs`` (stage + c2a), one Boolean matrix per pair fluent in symbol order, shape ``(entities, entities)`` + for a pair fluent between entities (an empty tuple in a world without pair fluents). The + arrays are read-only; ``apply`` returns a new state.""" values: np.ndarray numeric: np.ndarray = None # type: ignore[assignment] + pairs: tuple[np.ndarray, ...] = () def __post_init__(self) -> None: values = np.array(self.values, dtype=np.uint8, copy=True) @@ -87,6 +113,19 @@ def __post_init__(self) -> None: raise WorldError("a state holds finite numeric values only") numeric.setflags(write=False) object.__setattr__(self, "numeric", numeric) + pairs = [] + for matrix in self.pairs: + matrix = np.array(matrix, dtype=bool, copy=True) + if matrix.ndim != 2 or matrix.shape[0] != values.shape[0]: + raise WorldError( + f"the pairs of a state are 2-D matrices with one row per entity, got " + f"{matrix.shape} beside {values.shape}" + ) + if matrix.shape[0] == matrix.shape[1] and np.diagonal(matrix).any(): + raise WorldError("a pair fluent never holds between an entity and itself") + matrix.setflags(write=False) + pairs.append(matrix) + object.__setattr__(self, "pairs", tuple(pairs)) def __eq__(self, other: object) -> bool: if not isinstance(other, State): @@ -96,11 +135,22 @@ def __eq__(self, other: object) -> bool: and bool(np.array_equal(self.values, other.values)) and self.numeric.shape == other.numeric.shape and bool(np.array_equal(self.numeric, other.numeric)) + and len(self.pairs) == len(other.pairs) + and all( + a.shape == b.shape and bool(np.array_equal(a, b)) + for a, b in zip(self.pairs, other.pairs, strict=True) + ) ) def __hash__(self) -> int: return hash( - (self.values.shape, self.values.tobytes(), self.numeric.shape, self.numeric.tobytes()) + ( + self.values.shape, + self.values.tobytes(), + self.numeric.shape, + self.numeric.tobytes(), + tuple((m.shape, m.tobytes()) for m in self.pairs), + ) ) @classmethod @@ -109,12 +159,14 @@ def from_true( definition: RuntimeDefinition, true: Mapping[str, Iterable[str]], numeric: Mapping[str, Mapping[str, float]] | None = None, + pairs: Sequence[Sequence[str]] | None = None, ) -> State: """A state from the base fluents that are true, as ``{entity label: [fluent labels]}``, - and the numeric values as ``{entity label: {fluent label: value}}`` (the entities - table's initial values when None, and every entity and fluent must be given otherwise). - Every entity of the definition must be listed in ``true``; a fluent not listed is - false.""" + the numeric values as ``{entity label: {fluent label: value}}`` (the entities table's + initial values when None, and every entity and fluent must be given otherwise), and the + pairs that hold (stage c2a) as ``(pair fluent, first, second)`` label triples, every + ordered pair listed (the initial pairs of the definition when None). Every entity of + the definition must be listed in ``true``; a fluent not listed is false.""" values = np.zeros((definition.entity_count, len(definition.base_fluents)), dtype=np.uint8) missing = [label for label in definition.entity_labels if label not in true] if missing: @@ -127,11 +179,13 @@ def from_true( f"{fluent} is not a base fluent; a state holds base fluents only" ) values[row, definition.base_fluent_index(fluent)] = 1 + pair_matrices = _pairs_of(definition, pairs) if numeric is None: - return cls(values, _initial_numeric(definition)) + return cls(values, _initial_numeric(definition), pair_matrices) matrix = np.zeros((definition.entity_count, len(definition.numeric_fluents))) if not numeric: - return cls(values, matrix) # every numeric fluent at 0, for a caller that fills them + # every numeric fluent at 0, for a caller that fills them + return cls(values, matrix, pair_matrices) for i, label in enumerate(definition.entity_labels): entry = numeric.get(label) if entry is None: @@ -143,7 +197,17 @@ def from_true( for fluent in entry: if not definition.is_numeric_fluent(fluent): raise WorldError(f"{fluent} is not a numeric fluent of the definition") - return cls(values, matrix) + return cls(values, matrix, pair_matrices) + + def pairs_record(self, definition: RuntimeDefinition) -> list[list[str]]: + """Every pair that holds, as ``[pair fluent, first, second]`` label triples, by pair + fluent in symbol order, then by first and second in entity order; both directions of a + symmetric pair. The form of a fixture's ``pairs`` and a history's ``initial_pairs``.""" + _check_state(definition, self) + return [ + list(row) + for row in pair_rows(definition.pair_fluents, self.pairs, definition.entity_labels) + ] def true_fluents(self, definition: RuntimeDefinition) -> dict[str, list[str]]: """The base fluents that are true, as ``{entity label: [fluent labels]}``, every entity @@ -169,12 +233,39 @@ def _initial_numeric(definition: RuntimeDefinition) -> np.ndarray: return definition.initial_numeric +def _initial_pairs(definition: RuntimeDefinition) -> tuple[np.ndarray, ...]: + """The definition's initial pairs, or no pair at all when none were given (a fixture's + definition, whose initial pairs come apart from the record).""" + if not definition.pair_fluents: + return () + if definition.initial_pairs is None: + n = definition.entity_count + return tuple(np.zeros((n, n), dtype=bool) for _ in definition.pair_fluents) + return definition.initial_pairs + + +def _pairs_of( + definition: RuntimeDefinition, pairs: Sequence[Sequence[str]] | None +) -> tuple[np.ndarray, ...]: + """The pair matrices from label triples (the definition's initial pairs when None).""" + if pairs is None: + return _initial_pairs(definition) + if not definition.pair_fluents: + if list(pairs): + raise WorldError("the state lists pairs, but the definition has no pair fluents") + return () + entity_index = {label: i for i, label in enumerate(definition.entity_labels)} + return matrices_from_rows(pairs, definition.pair_fluents, entity_index, "the state's pairs") + + def initial_state(definition: RuntimeDefinition) -> State: """The state at ``TIME.1`` of every episode: the initial base fluents and numeric values of - the entities table.""" + the entities table, and the initial pairs of ``initial_pairs.csv`` (stage c2a).""" if definition.initial_values is None: raise WorldError("the definition's entities table carries no initial fluents") - return State(definition.initial_values, _initial_numeric(definition)) + return State( + definition.initial_values, _initial_numeric(definition), _initial_pairs(definition) + ) @dataclass(frozen=True) @@ -363,6 +454,18 @@ def _check_state(definition: RuntimeDefinition, state: State) -> None: f"the state's numeric part has shape {state.numeric.shape}; the definition needs " f"{numeric}" ) + if len(state.pairs) != len(definition.pair_fluents): + raise WorldError( + f"the state holds the pairs of {len(state.pairs)} pair fluents; the definition has " + f"{len(definition.pair_fluents)}" + ) + n = definition.entity_count + for fluent, matrix in zip(definition.pair_fluents, state.pairs, strict=True): + if matrix.shape != (n, n): + raise WorldError( + f"the pairs of {fluent.label} have shape {matrix.shape}; the definition needs " + f"{(n, n)}" + ) def _entity_rows(definition: RuntimeDefinition, entities: Sequence[int] | None) -> np.ndarray: @@ -394,6 +497,12 @@ def derive( elif reads["kind"] == "numeric_threshold": column = state.numeric[rows, definition.numeric_index(reads["fluent"])] literals[:, i] = (column > reads["threshold"]).astype(np.uint8) + elif reads["kind"] == PAIR_SOME: + matrix = state.pairs[definition.pair_index(reads["pair_fluent"])] + literals[:, i] = matrix[rows].any(axis=1).astype(np.uint8) + elif reads["kind"] == PAIR_SOMEOF: + matrix = state.pairs[definition.pair_index(reads["pair_fluent"])] + literals[:, i] = matrix[:, rows].any(axis=0).astype(np.uint8) outputs = definition.entity_matrices.evaluate(literals) fluent = ( np.stack([outputs[label] for label in definition.derived_fluents], axis=1) @@ -641,6 +750,10 @@ def check_definition_agreement( elif reads["kind"] == "numeric_threshold" and definition.initial_numeric is not None: column = state.numeric[:, definition.numeric_index(reads["fluent"])] literals[:, i] = (column > reads["threshold"]).astype(np.uint8) + elif reads["kind"] in (PAIR_SOME, PAIR_SOMEOF) and definition.initial_pairs is not None: + matrix = definition.initial_pairs[definition.pair_index(reads["pair_fluent"])] + axis = 1 if reads["kind"] == PAIR_SOME else 0 + literals[:, i] = matrix.any(axis=axis).astype(np.uint8) entity_report = check_agreement(definition.entity_matrices, definition.entity_rules, literals) n = definition.entity_count agents = np.repeat(np.arange(n), n) @@ -681,11 +794,15 @@ def _precondition_holds( facts = ( derive(definition, state, needed) if any( - not lit.numeric and not definition.is_base_fluent(lit.fluent) for lit in et.precondition + not lit.numeric and not lit.pair and not definition.is_base_fluent(lit.fluent) + for lit in et.precondition ) else None ) for lit in et.precondition: + if isinstance(lit, PairLiteral): + holds &= pair_literal_holds(definition, state, lit, et.roles, array) + continue rows = array[:, et.roles.index(lit.role)] if lit.numeric: column = state.numeric[rows, definition.numeric_index(lit.fluent)] @@ -701,6 +818,43 @@ def _precondition_holds( return holds +def pair_literal_holds( + definition: RuntimeDefinition, + state: State, + lit: PairLiteral, + roles: Sequence[str], + array: np.ndarray, +) -> np.ndarray: + """Whether a pair literal holds for each binding of ``array`` (one column per role) in the + state: ``PAIRFL.k(x, y)`` when the pair holds; ``SOME.PAIRFL.k(x)`` when the row of ``x`` + holds a pair; ``SOMEOF.PAIRFL.k(x)`` when its column does; ``SAME.PAIRFL.k(x, y)`` when + some ``z`` has both ``(x, z)`` and ``(y, z)``. A symmetric pair fluent holds both directions + of every pair, so a literal about it reads both. A negated literal holds when the positive + one does not.""" + matrix = state.pairs[definition.pair_index(lit.pair_fluent)] + first = array[:, roles.index(lit.first)] + if lit.kind == PAIR_KIND: + assert lit.second is not None + second = array[:, roles.index(lit.second)] + positive = matrix[first, second] + elif lit.kind == PAIR_SOME: + positive = matrix[first].any(axis=1) + elif lit.kind == PAIR_SOMEOF: + positive = matrix[:, first].any(axis=0) + else: + assert lit.second is not None + second = array[:, roles.index(lit.second)] + positive = (matrix[first] & matrix[second]).any(axis=1) + return positive if lit.value else ~positive + + +def _pair_literal_value( + definition: RuntimeDefinition, state: State, lit: PairLiteral, roles: Sequence[str], binding +) -> bool: + array = np.asarray([binding], dtype=np.intp) + return bool(pair_literal_holds(definition, state, lit, roles, array)[0]) + + def legal( definition: RuntimeDefinition, state: State, @@ -802,6 +956,13 @@ def _check_legal( if not _precondition_holds(definition, state, et, array)[0]: facts = derive(definition, state, list(event.binding)) for lit in et.precondition: + if isinstance(lit, PairLiteral): + if not _pair_literal_value(definition, state, lit, et.roles, event.binding): + raise IllegalEventError( + f"{text} is illegal: the precondition literal {lit.text} does not hold" + + (" (a compiled literal of the profile)" if lit.compiled else "") + ) + continue row = et.roles.index(lit.role) entity = event.binding[row] if lit.numeric: @@ -824,43 +985,120 @@ def _check_legal( raise AssertionError("the precondition fails but every literal holds") -def _writes(event: Event, et: EventTypeRecord) -> dict[tuple[int, str], bool | float]: - """The base fluents the event may write, ``{(entity, fluent): value}`` (a numeric effect's - signed constant for a numeric fluent): every effect counts, whether or not its condition - holds in the state.""" - return { - (event.binding[et.roles.index(effect.role)], effect.fluent): effect.value - for effect in et.effects - } +Key = tuple +"""A fact an event reads or writes: ``(entity, fluent)`` for a base Boolean or numeric fluent, +and ``(pair fluent, first, second)`` for one ordered pair of a pair fluent (stage c2a).""" + + +def _pair_keys( + definition: RuntimeDefinition, effect: Effect, event: Event, et: EventTypeRecord +) -> list[Key]: + """The ordered pairs a pair effect writes: the pair between its roles, and its reverse for a + symmetric pair fluent.""" + assert effect.second is not None + first = event.binding[et.roles.index(effect.role)] + second = event.binding[et.roles.index(effect.second)] + keys: list[Key] = [(effect.fluent, first, second)] + if definition.pair_fluent(effect.fluent).symmetric: + keys.append((effect.fluent, second, first)) + return keys + + +def _writes( + event: Event, et: EventTypeRecord, definition: RuntimeDefinition | None = None +) -> dict[Key, bool | float]: + """The facts the event may write, ``{key: value}`` (a numeric effect's signed constant for a + numeric fluent; an ordered pair, and its reverse for a symmetric pair fluent, for a pair + effect): every effect counts, whether or not its condition holds in the state.""" + found: dict[Key, bool | float] = {} + for effect in et.effects: + if effect.pair: + assert definition is not None + for key in _pair_keys(definition, effect, event, et): + found[key] = effect.value + else: + found[(event.binding[et.roles.index(effect.role)], effect.fluent)] = effect.value + return found + + +def _pair_reads( + definition: RuntimeDefinition, lit: PairLiteral, roles: Sequence[str], binding: Binding +) -> list[Key]: + """The ordered pairs a pair literal reads, every pair of a row or a column counted: the pair + (both directions for a symmetric pair fluent), the row of the role (``SOME``), the column + (``SOMEOF``), or the rows of both roles (``SAME``).""" + fluent = definition.pair_fluent(lit.pair_fluent) + first = binding[roles.index(lit.first)] + if lit.kind == PAIR_KIND: + assert lit.second is not None + second = binding[roles.index(lit.second)] + keys: list[Key] = [(fluent.label, first, second)] + if fluent.symmetric: + keys.append((fluent.label, second, first)) + return keys + if lit.kind == PAIR_SOME: + return _row_keys(definition, fluent.label, first) + if lit.kind == PAIR_SOMEOF: + return _column_keys(definition, fluent.label, first) + assert lit.second is not None + second = binding[roles.index(lit.second)] + return _row_keys(definition, fluent.label, first) + _row_keys(definition, fluent.label, second) + + +def _row_keys(definition: RuntimeDefinition, fluent: str, entity: int) -> list[Key]: + return [(fluent, entity, z) for z in range(definition.entity_count) if z != entity] + + +def _column_keys(definition: RuntimeDefinition, fluent: str, entity: int) -> list[Key]: + return [(fluent, z, entity) for z in range(definition.entity_count) if z != entity] + + +def _cone_reads(definition: RuntimeDefinition, fluent: str, entity: int) -> list[Key]: + """What a literal on a fluent of an entity reads: the base fluents of its cone, and, through + the pair literals of its rule, the rows and columns of the entity (stage c2a).""" + keys: list[Key] = [(entity, base) for base in definition.cone(fluent)] + for kind, pair_fluent in definition.pair_cone(fluent): + if kind == "row": + keys.extend(_row_keys(definition, pair_fluent, entity)) + else: + keys.extend(_column_keys(definition, pair_fluent, entity)) + return keys def _reads_by_reader( definition: RuntimeDefinition, event: Event, et: EventTypeRecord -) -> dict[tuple[int, str], str]: - """The base fluents the event's precondition and its effects' conditions read, through the - cone of every derived fluent: ``{(entity, base fluent): reader}``, the reader being ``"the - precondition"`` or ``"a condition"`` (the precondition when both read). A static literal of - a condition reads nothing that an event can write.""" - found: dict[tuple[int, str], str] = {} +) -> dict[Key, str]: + """The facts the event's precondition and its effects' conditions read, through the cone of + every derived fluent: ``{key: reader}``, the reader being ``"the precondition"`` or ``"a + condition"`` (the precondition when both read). A static literal of a condition reads + nothing that an event can write. A pair literal reads its pair (both directions of a + symmetric one), or every pair of the row or the column it names (stage c2a).""" + found: dict[Key, str] = {} for effect in et.effects: for lit in effect.condition: + if isinstance(lit, PairLiteral): + for key in _pair_reads(definition, lit, et.roles, event.binding): + found[key] = "a condition" + continue if lit.kind == "feature": continue entity = event.binding[et.roles.index(lit.role)] - for base in definition.cone(lit.symbol): - found[(entity, base)] = "a condition" + for key in _cone_reads(definition, lit.symbol, entity): + found[key] = "a condition" for lit in et.precondition: + if isinstance(lit, PairLiteral): + for key in _pair_reads(definition, lit, et.roles, event.binding): + found[key] = "the precondition" + continue entity = event.binding[et.roles.index(lit.role)] - for base in definition.cone(lit.fluent): - found[(entity, base)] = "the precondition" + for key in _cone_reads(definition, lit.fluent, entity): + found[key] = "the precondition" return found -def _reads( - definition: RuntimeDefinition, event: Event, et: EventTypeRecord -) -> set[tuple[int, str]]: - """The base fluents the event's precondition and its effects' conditions read, through the - cone of every derived fluent: ``{(entity, base fluent)}``.""" +def _reads(definition: RuntimeDefinition, event: Event, et: EventTypeRecord) -> set[Key]: + """The facts the event's precondition and its effects' conditions read, through the cone of + every derived fluent.""" return set(_reads_by_reader(definition, event, et)) @@ -878,6 +1116,10 @@ def _condition_holds( effect without a condition always holds.""" statics = static_facts(definition) for lit in effect.condition: + if isinstance(lit, PairLiteral): + if not _pair_literal_value(definition, state, lit, et.roles, event.binding): + return False + continue row = et.roles.index(lit.role) entity = event.binding[row] if lit.numeric: @@ -898,26 +1140,39 @@ def _condition_holds( return True +def _needs_derived(definition: RuntimeDefinition, et: EventTypeRecord) -> bool: + return any( + not lit.pair and lit.kind == "fluent" and not definition.is_base_fluent(lit.symbol) + for effect in et.effects + for lit in effect.condition + ) + + def _fired( definition: RuntimeDefinition, state: State, event: Event, et: EventTypeRecord -) -> dict[tuple[int, str], bool | float]: - """The base fluents the event writes in ``state``: its effects whose conditions hold there, - judged before any effect of the step (a numeric effect's value is its signed constant).""" +) -> dict[Key, bool | float]: + """The facts the event writes in ``state``: its effects whose conditions hold there, judged + before any effect of the step (a numeric effect's value is its signed constant; a pair + effect's both directions for a symmetric pair fluent).""" facts = None - if any( - lit.kind == "fluent" and not definition.is_base_fluent(lit.symbol) - for effect in et.effects - for lit in effect.condition - ): + if _needs_derived(definition, et): facts = derive(definition, state, list(event.binding)) - return { - (event.binding[et.roles.index(effect.role)], effect.fluent): effect.value - for effect in et.effects - if _condition_holds(definition, state, event, et, effect, facts) - } + found: dict[Key, bool | float] = {} + for effect in et.effects: + if not _condition_holds(definition, state, event, et, effect, facts): + continue + if effect.pair: + for key in _pair_keys(definition, effect, event, et): + found[key] = effect.value + else: + found[(event.binding[et.roles.index(effect.role)], effect.fluent)] = effect.value + return found -def _fact(definition: RuntimeDefinition, key: tuple[int, str]) -> str: +def _fact(definition: RuntimeDefinition, key: Key) -> str: + if len(key) == 3: + labels = definition.entity_labels + return f"{key[0]} between {labels[key[1]]} and {labels[key[2]]}" return f"{key[1]} of {definition.entity_labels[key[0]]}" @@ -929,7 +1184,7 @@ def interferes(definition: RuntimeDefinition, a: Event, b: Event) -> bool: if a == b: return True et_a, et_b = definition.event_type(a.event_type), definition.event_type(b.event_type) - writes_a, writes_b = set(_writes(a, et_a)), set(_writes(b, et_b)) + writes_a, writes_b = set(_writes(a, et_a, definition)), set(_writes(b, et_b, definition)) if writes_a & writes_b: return True return bool(writes_a & _reads(definition, b, et_b)) or bool( @@ -971,7 +1226,7 @@ def apply_step( types = [_check_event(definition, event) for event in events] for event, et in zip(events, types, strict=True): _check_legal(definition, state, event, et) - writes = [_writes(event, et) for event, et in zip(events, types, strict=True)] + writes = [_writes(event, et, definition) for event, et in zip(events, types, strict=True)] reads = [ _reads_by_reader(definition, event, et) for event, et in zip(events, types, strict=True) ] @@ -997,9 +1252,14 @@ def apply_step( ) values = np.array(state.values, dtype=np.uint8, copy=True) numeric = np.array(state.numeric, dtype=np.float64, copy=True) + pairs = [np.array(matrix, dtype=bool, copy=True) for matrix in state.pairs] low, high = definition.range_bounds for event, et in zip(events, types, strict=True): - for (entity, fluent), value in _fired(definition, state, event, et).items(): + for key, value in _fired(definition, state, event, et).items(): + if len(key) == 3: + pairs[definition.pair_index(key[0])][key[1], key[2]] = bool(value) + continue + entity, fluent = key if definition.is_numeric_fluent(fluent): j = definition.numeric_index(fluent) numeric[entity, j] = clamp( @@ -1010,7 +1270,29 @@ def apply_step( after_effects = np.array(numeric, copy=True) if numeric.shape[1] and drift: numeric = np.minimum(np.maximum(numeric + definition.drift_vector[None, :], low), high) - return StepResult(State(values, numeric), after_effects) + _check_profiles(definition, pairs, events) + return StepResult(State(values, numeric, tuple(pairs)), after_effects) + + +def _check_profiles( + definition: RuntimeDefinition, pairs: Sequence[np.ndarray], events: Sequence[Event] +) -> None: + """Every profile holds after the step; otherwise :class:`ProfileViolationError` names the + pair fluent, the pairs, and the events. The compiled literals (F4) make a violation + impossible for a correct definition.""" + for fluent, matrix in zip(definition.pair_fluents, pairs, strict=True): + violations = profile_violations(fluent, matrix) + if violations: + labels = definition.entity_labels + held = ", ".join( + f"({labels[int(i)]}, {labels[int(j)]})" + for i, j in zip(*np.nonzero(matrix), strict=True) + ) + raise ProfileViolationError( + f"after the step the profile of {fluent.label} does not hold: {violations[0]}; " + f"its pairs are {held or 'none'}; the events were " + f"{', '.join(e.describe(definition) for e in events) or 'none'}" + ) def fired_effects(definition: RuntimeDefinition, state: State, event: Event) -> tuple[Effect, ...]: @@ -1267,6 +1549,7 @@ def decrease( "State", "StaticFacts", "StepResult", + "Key", "able", "able_table", "above", @@ -1286,6 +1569,7 @@ def decrease( "legal", "legal_bindings", "is_place_class", + "pair_literal_holds", "members", "place_able_table", "place_categories", diff --git a/python/semantic_world/world/stats.py b/python/semantic_world/world/stats.py index 1c4a52d..5698715 100644 --- a/python/semantic_world/world/stats.py +++ b/python/semantic_world/world/stats.py @@ -114,7 +114,7 @@ def absorbing_fluents(event_types: EventTypes, fluents: Fluents) -> list[str]: set_to: dict[str, set[bool]] = {f.label: set() for f in fluents.base} for et in event_types.leaves: for effect in et.effects: - if not effect.numeric: + if not effect.numeric and not effect.pair: set_to[effect.fluent].add(effect.value) return [label for label, values in set_to.items() if len(values) == 1] @@ -329,7 +329,12 @@ def world_stats( conditional[kind] += 1 for lit in effect.condition: condition_literals["total"] += 1 - condition_literals[lit.kind] += 1 + # a pair condition literal (stage c2a) counts under pair, written only when + # the world has one, so that a world without pair fluents writes its statistics + # as before + condition_literals["pair" if lit.pair else lit.kind] = ( + condition_literals.get("pair" if lit.pair else lit.kind, 0) + 1 + ) condition_literals[lit.role] += 1 edges = enabling_edges(event_types) return { @@ -366,6 +371,11 @@ def world_stats( "feature": condition_literals["feature"], "fluent": condition_literals["fluent"], "numeric": condition_literals["numeric"], + **( + {"pair": condition_literals["pair"]} + if condition_literals.get("pair") + else {} + ), }, "by_role": { "agent": condition_literals["agent"], diff --git a/python/semantic_world/world/streams.py b/python/semantic_world/world/streams.py index f53cfad..8f91673 100644 --- a/python/semantic_world/world/streams.py +++ b/python/semantic_world/world/streams.py @@ -17,7 +17,10 @@ whose preconditions are redrawn draws from its own parts, ``world:preconditions: