From bfae3c419ff98ab0986c31bac18fce20be2d9ea2 Mon Sep 17 00:00:00 2001 From: Sander Striker Date: Thu, 12 Nov 2026 13:58:52 +0100 Subject: [PATCH 1/4] feat: give a rule a lifecycle, an owner and a denial that says what would pass This is package 6 of PLAN.md, section 7 of the paper. - The policy block has `status`, `review_by`, `core` and `signed_off_by`. A rule is proposed, advisory, enforced or retired. A rule is in a decision only when a person verified the version that says so. The policy loads no module from an entry below verified. - An advisory rule reports what it would refuse and refuses nothing. The decision and the decision log have `advisories` and `denials`. - A denial can give a reason code and a remedy. The refusal shows them. - `[gates] max_enforced_rules` is a ceiling on the enforced rules of one domain. A rule over the ceiling is advisory until a person signs it off. - Only an L0 rule in the marked core takes no exception. - The memory tools `check_policy`, `file_exception` and `propose_amendment`, and the commands `amem policy check`, `except`, `amend` and `reset`. The reset retires each rule of a domain that no person wrote, promoted or approved. - The lint reports a rule with no owner, a rule after its review date, a rule over the ceiling and an unused rule. - Act 3 shows the steps, a refusal, an amendment, an exception and a reset. Co-Authored-By: Claude Fable 5.2 --- PLAN.md | 3 +- README.md | 9 +- crates/memory/src/main.rs | 138 ++++++++++++- crates/policy/rego/amem.rego | 24 ++- crates/policy/src/decide.rs | 246 +++++++++++++++++++++++- crates/policy/src/lib.rs | 6 + crates/policy/src/record.rs | 188 +++++++++++++++++- crates/server/src/lib.rs | 278 ++++++++++++++++++++++++++- crates/server/src/mcp.rs | 157 +++++++++++++++ crates/server/src/rules.rs | 362 +++++++++++++++++++++++++++++++++++ docs/demo-narrative.md | 38 +++- docs/terms.md | 5 + local/README.md | 88 +++++++-- scripts/demo/3-perimeter.sh | 85 ++++---- 14 files changed, 1546 insertions(+), 81 deletions(-) create mode 100644 crates/server/src/rules.rs diff --git a/PLAN.md b/PLAN.md index 05a58389..381e8aab 100644 --- a/PLAN.md +++ b/PLAN.md @@ -69,7 +69,8 @@ Landed so far: | 0 | #69 | Nothing | | 1 | #71 | Nothing | | 2 | #73 | An attempt key on every operation; an uncertain external effect blocking reassignment; waiting on an outside event as a state; the record's remaining fields (`parent`, `scores`, `implements`, `reversibility`, `artifacts_touched`, `policy_context`) | -| 3 | the pull request after #73 | A suite that gates a version of a definition; the data scope enforced; a definition required by default, once the system's own filings name one; tools offered a few at boot and the rest by search | +| 3 | #74 | A suite that gates a version of a definition; the data scope enforced; a definition required by default, once the system's own filings name one; tools offered a few at boot and the rest by search | +| 6 | this pull request | `check_policy` selecting rules by the action; the unused-rule check over one review period; the approver of an exception checked against the layer breached; the ceiling counting rules by age; a session that reloads the policy when a rule changes; a verified rule that stays in force while its next version waits for review | What stays unbuilt after these, and stays on README's list: federation across domains, authentication of people, logprob-based drift, and accelerator time, which only a gateway can count. So do the items on that list that need another machine or an outside service more than they need code: the limits on re-enactment, harness settings and decomposition templates in the platform domain, bytes stored and resource limits on the sandbox, the authoring of agent definitions, the search tool against its live provider, a harness under Seatbelt on a Mac, and the Gofannon bridge against a live deployment, whose agents make model calls that the egress does not meter. diff --git a/README.md b/README.md index 2b102984..2a1f2503 100644 --- a/README.md +++ b/README.md @@ -123,9 +123,12 @@ two weeks. It is also the reason that none of it is the production shape. - **Approval events.** An approval is the completion of a promotion task. It has no scope and no expiry, and the perimeter does not check it when an action runs. There is no `declassify` and no guard. -- **The rule lifecycle.** A rule is enforced when a person verified it and it has a - module. A rule has no status, no owner who is a person, and no review date that the - policy uses. There is no ceiling on rules, no reset, and no `check_policy` tool. +- **The rule lifecycle, in part.** `check_policy` gives all the rules in force for a + domain. It does not select rules by the action. The lint reports a rule after its + review date and an unused rule, from the full decision log and not from one review + period. The approver of an exception is a name in the answer. `amem` does not check + that this person approves. The order of rules under the ceiling is the order of their + ids. While the next version of a rule waits for its review, the rule is in no decision. - **Lineage, in part.** An entry does not record its session. `amem taint` compares the time of a read with the time of a later write, and it goes forward only. Reads and sends are not in the event log. A write that the scan refuses is a file under `state/`, diff --git a/crates/memory/src/main.rs b/crates/memory/src/main.rs index 0653862e..59800446 100644 --- a/crates/memory/src/main.rs +++ b/crates/memory/src/main.rs @@ -1067,6 +1067,47 @@ enum PolicyOp { #[arg(long)] file: bool, }, + /// The rules in force for an action in a domain, enforced and advisory. + Check { + /// What is about to be done, in a few words. + action: String, + #[arg(long)] + domain: Option, + }, + /// Ask for an exception to a rule, for one task, until an expiry. A person approves + /// it with `amem approve`. + Except { + /// The policy id of the rule to breach. + breaches: String, + /// The one task the exception holds for. + #[arg(long)] + task: String, + /// What it covers, in words. + #[arg(long)] + scope: String, + /// When it lapses, RFC 3339. + #[arg(long)] + expiry: String, + /// Why it is needed. + #[arg(long)] + justification: String, + #[arg(long)] + domain: Option, + }, + /// Propose an amendment to a rule: a task for the rule's owner, with the evidence. + Amend { + /// The policy id of the rule. + policy_id: String, + /// What happened, and the ids of the entries that show it. + #[arg(long)] + evidence: String, + }, + /// Retire every rule of a domain that no person wrote or promoted, and keep the + /// rest. The owner's act. + Reset { + #[arg(long)] + domain: Option, + }, } #[derive(Subcommand)] @@ -4589,6 +4630,7 @@ async fn perimeter_cmd(m: &Memory, command: Command) -> Fallible { Command::Policy { op: PolicyOp::Lint { file }, } => policy_lint(m, file).await, + Command::Policy { op } => rule_cmd(m, op).await, Command::Quarantine { op: QuarantineOp::List, } => json( @@ -4648,7 +4690,11 @@ fn policy_list(m: &Memory) -> serde_json::Value { serde_json::json!({ "id": e.meta.id, "entry": e.entry_id, "layer": e.meta.layer, "tier": e.tier, "rule": e.meta.rule, "exception": e.meta.is_exception(), - "enforced": e.enforced(), "in_force": e.in_force(now), + "enforced": e.enforced() && p.state_of(e) == dm_policy::record::Status::Enforced, + "in_force": e.in_force(now), + "status": if e.meta.is_exception() { "exception" } else { p.state_of(e).as_str() }, + "owner": e.meta.owner, + "review_by": e.meta.review_by, "breaches": e.meta.breaches, "scope_task": e.meta.scope_task, }) }) @@ -4663,7 +4709,7 @@ fn policy_list(m: &Memory) -> serde_json::Value { /// refused something, and how often an exception let something through. fn policy_rates(root: &Path) -> serde_json::Value { let text = std::fs::read_to_string(root.join("state/decisions.jsonl")).unwrap_or_default(); - let mut cited: std::collections::BTreeMap = + let mut cited: std::collections::BTreeMap = std::collections::BTreeMap::new(); for line in text.lines() { let Ok(v) = serde_json::from_str::(line) else { @@ -4683,13 +4729,98 @@ fn policy_rates(root: &Path) -> serde_json::Value { row.0 += 1; } } + // what an advisory rule would have refused, had it been enforced + for id in v["advisories"] + .as_array() + .into_iter() + .flatten() + .filter_map(|a| a["policy"].as_str()) + { + cited.entry(id.to_string()).or_default().2 += 1; + } } serde_json::json!(cited .into_iter() - .map(|(id, (refused, overridden))| serde_json::json!({"id": id, "refused": refused, "let_through": overridden})) + .map(|(id, (refused, overridden, advised))| serde_json::json!({"id": id, "refused": refused, "let_through": overridden, "advised": advised})) .collect::>()) } +/// The findings that need the grants and the decision log: a rule held at advisory by +/// the ceiling on enforced rules, and an enforced rule that no decision in the log +/// cites, which goes to its owner to retire or to say why it stays. +fn ceiling_and_use(m: &Memory, p: &dm_policy::decide::Policy) -> Vec { + let mut out = Vec::new(); + for id in p.over_ceiling(m.grants()) { + out.push(dm_policy::record::Finding { + why: format!( + "{id} is over the ceiling on enforced rules and is advisory: a person signs this one rule off with signed_off_by, or a rule is retired" + ), + policy: id, + kind: "over-ceiling".into(), + }); + } + let rates = policy_rates(m.root()); + let cited: Vec<&str> = rates + .as_array() + .into_iter() + .flatten() + .filter_map(|r| r["id"].as_str()) + .collect(); + for e in p.entries().iter().filter(|e| e.enforced()) { + if !cited.contains(&e.meta.id.as_str()) { + out.push(dm_policy::record::Finding { + policy: e.meta.id.clone(), + kind: "unused".into(), + why: format!( + "{} is enforced and no decision in the log cites it: its owner retires it, or says why it stays", + e.meta.id + ), + }); + } + } + out +} + +/// `amem policy check`, `except`, `amend` and `reset`: the rule lifecycle. +async fn rule_cmd(m: &Memory, op: PolicyOp) -> Fallible { + match op { + PolicyOp::Check { action, domain } => { + json(&serde_json::json!({ "rules": m.check_policy(&action, domain.as_deref())? })) + } + PolicyOp::Except { + breaches, + task, + scope, + expiry, + justification, + domain, + } => { + let ask = dm_server::rules::ExceptionRequest { + breaches, + task: Some(task), + scope, + expires: expiry, + justification, + domain, + }; + json(&m.file_exception(ask).await?) + } + PolicyOp::Amend { + policy_id, + evidence, + } => { + let created = m.propose_amendment(&policy_id, &evidence).await?; + json(&serde_json::json!({ + "task": created.task, "deduplicated": created.deduplicated, + })) + } + PolicyOp::Reset { domain } => json(&m.reset_rules(domain.as_deref()).await?), + PolicyOp::Show | PolicyOp::Eval { .. } | PolicyOp::List | PolicyOp::Lint { .. } => { + unreachable!("routed by perimeter_cmd") + } + } +} + /// `amem policy lint`. async fn policy_lint(m: &Memory, file: bool) -> Fallible { let p = m.policy(); @@ -4703,6 +4834,7 @@ async fn policy_lint(m: &Memory, file: bool) -> Fallible { why: format!("{id}'s module would not load: {why}"), }), ); + findings.extend(ceiling_and_use(m, &p)); let mut filed = Vec::new(); if file { let domain = m.resolve_domain(None)?; diff --git a/crates/policy/rego/amem.rego b/crates/policy/rego/amem.rego index 79b86a1f..061bb836 100644 --- a/crates/policy/rego/amem.rego +++ b/crates/policy/rego/amem.rego @@ -373,7 +373,15 @@ base_decision := erase_decision if input.action == "erase" core_policy(pid) if { some p in data.policies p.id == pid - p.layer == 0 + p.core == true +} + +# An advisory rule is evaluated and reported, and refuses nothing. A rule the data does +# not list, a built-in one, is enforced. +advisory_policy(pid) if { + some p in data.policies + p.id == pid + p.status == "advisory" } excepted(pid) if { @@ -391,8 +399,18 @@ all_denials contains d if some d in builtin_denials rule_denials contains d if { some d in all_denials not excepted(d.policy) + not advisory_policy(d.policy) } +advisory_denials contains d if { + some d in all_denials + advisory_policy(d.policy) +} + +advisory_list := [d | some d in advisory_denials] + +denial_list := [d | some d in rule_denials] + exceptions_used contains x if { some d in all_denials excepted(d.policy) @@ -408,7 +426,7 @@ denied_list := [p | some d in rule_denials; p := d.policy] denied_reasons := [r | some d in rule_denials; r := sprintf("%v: %v", [d.policy, d.reason])] -decision := object.union(base_decision, {"cites": used_list}) if { +decision := object.union(base_decision, {"cites": used_list, "advisories": advisory_list}) if { count(rule_denials) == 0 } @@ -418,6 +436,8 @@ decision := { "review": "human", "reason": concat("; ", denied_reasons), "cites": denied_list, + "denials": denial_list, + "advisories": advisory_list, } if { count(rule_denials) > 0 } diff --git a/crates/policy/src/decide.rs b/crates/policy/src/decide.rs index b71298d2..9aab9657 100644 --- a/crates/policy/src/decide.rs +++ b/crates/policy/src/decide.rs @@ -248,6 +248,74 @@ pub struct Decision { /// The digest of the policy text that decided. #[serde(default)] pub policy: String, + /// Each rule that refused, with what a caller needs to act on the refusal: the + /// rule, why, a reason code, and what would pass. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub denials: Vec, + /// What the advisory rules would have refused, had they been enforced. They refuse + /// nothing: this is how a rule is watched before it is given teeth. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub advisories: Vec, +} + +/// One rule's refusal, as its module states it. +#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub struct Denial { + /// The rule. + pub policy: String, + /// Why, in a sentence. + #[serde(default)] + pub reason: String, + /// A short code for the reason, the same for every refusal of its kind. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub code: Option, + /// What would pass. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub remedy: Option, +} + +impl Denial { + /// The refusal in a sentence a caller can act on: the rule, why, the code, and what + /// would pass. + #[must_use] + pub fn said(&self) -> String { + let mut out = format!("{}: {}", self.policy, self.reason); + if let Some(code) = &self.code { + out.push_str(&format!(" (code {code})")); + } + if let Some(remedy) = &self.remedy { + out.push_str(&format!(". To pass: {}.", remedy.trim_end_matches('.'))); + } + out + } +} + +/// The enforced rules over a ceiling, which is a domain's: in each domain, counted in +/// the order of their ids, each one past the ceiling that carries no sign-off of its +/// own. A signed-off rule is enforced and takes no place under the ceiling. +fn over_ceiling(entries: &[crate::record::Entry], ceiling: Option) -> Vec { + let Some(ceiling) = ceiling else { + return Vec::new(); + }; + let mut by_domain: std::collections::BTreeMap<&str, Vec<&str>> = + std::collections::BTreeMap::new(); + for e in entries + .iter() + .filter(|e| e.enforced() && e.meta.signed_off_by.is_none()) + { + by_domain + .entry(e.domain.as_str()) + .or_default() + .push(e.meta.id.as_str()); + } + let mut over = Vec::new(); + for counted in by_domain.values_mut() { + counted.sort_unstable(); + counted.dedup(); + over.extend(counted.iter().skip(ceiling).map(|id| (*id).to_string())); + } + over.sort(); + over } /// What a session may read, as the policy computes it: the conditions an entry must @@ -296,6 +364,8 @@ pub struct Policy { entries: Vec, /// Modules a verified entry carried that would not load, and why. refused: Vec<(String, String)>, + /// The rules the ceiling holds at advisory. + held_back: Vec, } impl std::fmt::Debug for Policy { @@ -367,7 +437,10 @@ impl Policy { now: time::OffsetDateTime, ) -> Result { let mut texts = self.text.clone(); - for e in entries.iter().filter(|e| e.enforced()) { + let over = over_ceiling(&entries, grants.gates.max_enforced_rules); + // an advisory rule's module is loaded with the enforced ones: it is evaluated + // with every decision, and the base reports what it says without refusing + for e in entries.iter().filter(|e| e.enforced() || e.advisory()) { let Some(rego) = &e.rego else { continue }; if !crate::record::module_is_confined(rego) { self.refused @@ -386,7 +459,18 @@ impl Policy { let policies: Vec<_> = entries .iter() .filter(|e| !e.meta.is_exception()) - .map(|e| serde_json::json!({"id": e.meta.id, "layer": e.meta.layer})) + .map(|e| { + // a rule over the ceiling that nobody signed off is held at advisory + let status = if over.contains(&e.meta.id) { + crate::record::Status::Advisory + } else { + e.state() + }; + serde_json::json!({ + "id": e.meta.id, "layer": e.meta.layer, + "core": e.meta.is_core(), "status": status.as_str(), + }) + }) .collect(); let exceptions: Vec<_> = entries .iter() @@ -403,9 +487,28 @@ impl Policy { self.engine = Policy::with_data(self.engine, &data)?; self.version = digest(&texts); self.entries = entries; + self.held_back = over; Ok(self) } + /// Where a rule is in its life under this policy: as its entry says, or advisory + /// when the ceiling on enforced rules holds it back. + #[must_use] + pub fn state_of(&self, rule: &crate::record::Entry) -> crate::record::Status { + if self.held_back.contains(&rule.meta.id) { + crate::record::Status::Advisory + } else { + rule.state() + } + } + + /// The rules held at advisory because the grants' ceiling on enforced rules is + /// reached and no person signed them off. + #[must_use] + pub fn over_ceiling(&self, grants: &Grants) -> Vec { + over_ceiling(&self.entries, grants.gates.max_enforced_rules) + } + fn with_data( mut engine: regorus::Engine, data: &serde_json::Value, @@ -451,6 +554,7 @@ impl Policy { source: source.to_string(), entries: Vec::new(), refused: Vec::new(), + held_back: Vec::new(), }) } @@ -506,6 +610,10 @@ impl Policy { let mut d: Decision = serde_json::from_str(&json).map_err(|e| Error::Policy(format!("{e}: {json}")))?; d.policy = self.version.clone(); + if !d.allow && !d.denials.is_empty() { + let said: Vec = d.denials.iter().map(Denial::said).collect(); + d.reason = said.join("; "); + } Ok(d) } } @@ -838,6 +946,140 @@ mod tests { .unwrap() } + fn with(g: &Grants, entries: Vec) -> Policy { + Policy::builtin(g) + .unwrap() + .with_record(g, entries, time::OffsetDateTime::now_utc()) + .unwrap() + } + + /// A rule's life: proposed and retired rules are in no decision, an advisory rule is + /// reported and refuses nothing, and an enforced one refuses. + #[test] + fn a_rule_refuses_only_once_enforced_and_is_watched_while_advisory() { + use dm_core::TrustTier::{Curated, Raw, Verified}; + let g = grants(); + let rule = |tier, status: &str| { + rule_entry( + "pol-L2-031", + 2, + tier, + &format!("status = \"{status}\""), + true, + ) + }; + // below verified a rule is in no decision: nobody's module is evaluated that a + // person has not verified + for (tier, status) in [ + (Raw, "advisory"), + (Raw, "enforced"), + (Curated, "advisory"), + (Curated, "enforced"), + (Verified, "proposed"), + (Verified, "retired"), + ] { + let d = supersede_decision(&with(&g, vec![rule(tier, status)]), None); + assert!( + d.allow && d.advisories.is_empty(), + "{status} at {tier}: {d:?}" + ); + } + let d = supersede_decision(&with(&g, vec![rule(Verified, "advisory")]), None); + assert!(d.allow, "{d:?}"); + assert_eq!(d.advisories.len(), 1, "{d:?}"); + assert_eq!(d.advisories[0].policy, "pol-L2-031"); + assert!(d.cites.is_empty() && d.denials.is_empty(), "{d:?}"); + let d = supersede_decision(&with(&g, vec![rule(Verified, "enforced")]), None); + assert!(!d.allow && d.advisories.is_empty(), "{d:?}"); + } + + /// A refusal says what would pass: the rule's module gives a code and a remedy, and + /// the decision carries them. + #[test] + fn a_denial_carries_the_rule_s_reason_code_and_remedy() { + let g = grants(); + let body = "```policy\nid = \"pol-L2-040\"\nlayer = 2\n```\n\n```rego\npackage amem.rules\nimport rego.v1\ndeny contains {\"policy\": \"pol-L2-040\", \"reason\": \"an agent superseded a decision\", \"code\": \"decision-by-agent\", \"remedy\": \"propose the change to the decision's owner\"} if {\n\tinput.principal.class == \"agent\"\n\tinput.write.supersedes_type == \"decision\"\n}\n```\n"; + let rule = crate::record::parse("mem-p-40", dm_core::TrustTier::Verified, body).unwrap(); + let d = supersede_decision(&with(&g, vec![rule]), None); + assert!(!d.allow); + assert_eq!( + d.denials, + vec![Denial { + policy: "pol-L2-040".into(), + reason: "an agent superseded a decision".into(), + code: Some("decision-by-agent".into()), + remedy: Some("propose the change to the decision's owner".into()), + }] + ); + } + + /// The ceiling: the rules past it, in the order of their ids, are held at advisory + /// until a person signs one off, and a signature admits that rule alone. + #[test] + fn a_rule_over_the_ceiling_is_advisory_until_a_person_signs_it_off() { + use dm_core::TrustTier::Verified; + let mut g = grants(); + g.gates.max_enforced_rules = Some(1); + // two rules that refuse the same thing; a third that never fires takes a place + let first = rule_entry("pol-L2-001", 2, Verified, "", true); + let second = rule_entry("pol-L2-002", 2, Verified, "", true); + let p = with(&g, vec![second.clone(), first.clone()]); + assert_eq!(p.over_ceiling(&g), ["pol-L2-002"]); + let d = supersede_decision(&p, None); + assert_eq!(d.cites, ["pol-L2-001"], "the first refuses: {d:?}"); + assert_eq!(d.advisories.len(), 1); + assert_eq!(d.advisories[0].policy, "pol-L2-002"); + // signed off, the second is enforced, and takes no place under the ceiling + let signed = rule_entry( + "pol-L2-002", + 2, + Verified, + "signed_off_by = \"human:test:lead\"", + true, + ); + let third = rule_entry("pol-L2-003", 2, Verified, "", true); + let p = with(&g, vec![first, signed, third]); + assert_eq!( + p.over_ceiling(&g), + ["pol-L2-003"], + "the signature raised nothing" + ); + let mut cites = supersede_decision(&p, None).cites; + cites.sort(); + assert_eq!(cites, ["pol-L2-001", "pol-L2-002"]); + assert_eq!(p.state_of(&p.entries()[2]), crate::record::Status::Advisory); + assert_eq!(p.state_of(&p.entries()[0]), crate::record::Status::Enforced); + // the ceiling is a domain's: the same rules in two domains are each under it + let mut elsewhere = rule_entry("pol-L2-004", 2, Verified, "", true); + elsewhere.domain = "dom-other".into(); + let here = rule_entry("pol-L2-001", 2, Verified, "", true); + let apart = with(&g, vec![here, elsewhere]); + assert!(apart.over_ceiling(&g).is_empty()); + // with no ceiling, all three refuse + g.gates.max_enforced_rules = None; + assert!(p.over_ceiling(&g).is_empty()); + } + + /// The core of L0 takes no exception; an L0 rule marked as outside the core takes one. + #[test] + fn only_the_marked_core_of_l0_takes_no_exception() { + use dm_core::TrustTier::Verified; + let g = grants(); + let exc = rule_entry( + "exc-2026-003", + 0, + Verified, + "breaches = \"pol-L0-002\"\nscope_task = \"task-t-1\"\nexpires = \"2099-01-01T00:00:00Z\"", + false, + ); + let outside = rule_entry("pol-L0-002", 0, Verified, "core = false", true); + let p = with(&g, vec![outside, exc.clone()]); + assert!(supersede_decision(&p, Some("task-t-1")).allow); + let core = rule_entry("pol-L0-002", 0, Verified, "core = true", true); + let p = with(&g, vec![core, exc]); + assert!(!supersede_decision(&p, Some("task-t-1")).allow); + } + #[test] fn a_verified_rule_from_the_record_refuses_and_a_scoped_exception_lets_one_task_through() { use dm_core::TrustTier::{Raw, Verified}; diff --git a/crates/policy/src/lib.rs b/crates/policy/src/lib.rs index 03b8f8ad..c6e51c48 100644 --- a/crates/policy/src/lib.rs +++ b/crates/policy/src/lib.rs @@ -85,6 +85,11 @@ pub struct Gates { /// for itself name no definition yet. #[serde(default, skip_serializing_if = "is_false")] pub require_definition: bool, + /// The most rules that are enforced on the strength of their verification alone. + /// A rule over the ceiling, counted in the order of the rules' ids, is advisory + /// until a person signs that one rule off in its block. None is no ceiling. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub max_enforced_rules: Option, } #[allow(clippy::trivially_copy_pass_by_ref)] // serde passes a reference @@ -106,6 +111,7 @@ impl Default for Gates { max_supersedes_per_session: default_max_supersedes(), verified_needs_approver: default_true(), require_definition: false, + max_enforced_rules: None, } } } diff --git a/crates/policy/src/record.rs b/crates/policy/src/record.rs index 84b315ac..1480d09a 100644 --- a/crates/policy/src/record.rs +++ b/crates/policy/src/record.rs @@ -27,6 +27,36 @@ use serde::{Deserialize, Serialize}; +/// Where a rule is in its life. Each step up is a reviewed promotion: a rule is in a +/// decision, as advisory or as enforced, only once a person has verified the version +/// that says so. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum Status { + /// Written, and not yet in any decision. + Proposed, + /// Evaluated with every decision and reported, refusing nothing: where a rule + /// written after something went wrong starts. + Advisory, + /// Refuses. + Enforced, + /// No longer in any decision. Kept, as every entry is. + Retired, +} + +impl Status { + /// The word the block and the data hold. + #[must_use] + pub fn as_str(self) -> &'static str { + match self { + Status::Proposed => "proposed", + Status::Advisory => "advisory", + Status::Enforced => "enforced", + Status::Retired => "retired", + } + } +} + /// What the `policy` block says. #[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] pub struct Meta { @@ -44,9 +74,33 @@ pub struct Meta { /// The policy ids this one implements; an L3 convention names the L2 it serves. #[serde(default)] pub implements: Vec, - /// The domain that owns it. + /// Who owns it: the person who answers for the rule, as `human::`, or + /// the domain that does. #[serde(default)] pub owner: Option, + /// Where the rule is in its life. A block that does not say is enforced once its + /// entry is verified, as before the lifecycle. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub status: Option, + /// When the rule must be looked at again, a calendar date. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub review_by: Option, + /// For an L0 rule: whether it is part of the core, which takes no exception. An L0 + /// rule that does not say is. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub core: Option, + /// The person who signed this rule off above the domain's ceiling on enforced rules. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub signed_off_by: Option, + /// For an exception: what it covers, in words. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub scope: Option, + /// For an exception: who approved it. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub approver: Option, + /// For an exception: why it is needed. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub justification: Option, /// For an exception: the policy id it breaches. #[serde(default)] pub breaches: Option, @@ -129,6 +183,12 @@ impl Meta { pub fn is_exception(&self) -> bool { self.breaches.is_some() } + + /// Whether this is an L0 rule of the core, which takes no exception. + #[must_use] + pub fn is_core(&self) -> bool { + self.layer == 0 && self.core.unwrap_or(true) + } } /// A policy entry as the record holds it. @@ -142,15 +202,39 @@ pub struct Entry { pub meta: Meta, /// Its Rego module, if it carries one. pub rego: Option, + /// The domain its entry is in; empty for one that came from no record. + #[serde(default)] + pub domain: String, } impl Entry { - /// Whether its module is enforced: verified and carrying one. + /// Where the rule is in its life, as its block and its entry's tier say together. + /// The block names the step the rule's author means it to be at, and the tier + /// holds it back: below verified a rule is proposed, whatever its block says, so no + /// module is evaluated that a person has not verified. A module's denial names the + /// rule it speaks for, and one evaluated unreviewed could name another rule's. A + /// block that names no status is enforced once verified. + #[must_use] + pub fn state(&self) -> Status { + use dm_core::TrustTier::Verified; + match (self.meta.status, self.tier) { + (Some(Status::Retired), _) => Status::Retired, + (Some(Status::Enforced) | None, Verified) => Status::Enforced, + (Some(Status::Advisory), Verified) => Status::Advisory, + _ => Status::Proposed, + } + } + + /// Whether its module refuses: an enforced rule carrying one. #[must_use] pub fn enforced(&self) -> bool { - self.tier == dm_core::TrustTier::Verified - && self.rego.is_some() - && !self.meta.is_exception() + self.state() == Status::Enforced && self.rego.is_some() && !self.meta.is_exception() + } + + /// Whether its module is evaluated and reported without refusing. + #[must_use] + pub fn advisory(&self) -> bool { + self.state() == Status::Advisory && self.rego.is_some() && !self.meta.is_exception() } /// Whether, as an exception, it is in force at `now`: verified, not expired. @@ -189,9 +273,38 @@ pub fn parse(entry_id: &str, tier: dm_core::TrustTier, body: &str) -> Result Result { + let block = fenced(body, "policy").ok_or("no ```policy block")?; + let mut table: toml::Table = toml::from_str(block).map_err(|e| format!("policy block: {e}"))?; + table.insert("status".into(), toml::Value::String(status.as_str().into())); + let new = toml::to_string(&table).map_err(|e| format!("policy block: {e}"))?; + Ok(body.replacen(block, new.trim_end(), 1)) +} + +/// The body of a policy entry: a heading, what a person reads, and the block. +/// +/// # Errors +/// +/// A block that will not be written as TOML. +pub fn render(title: &str, text: &str, meta: &Meta) -> Result { + let block = toml::to_string(meta).map_err(|e| format!("policy block: {e}"))?; + Ok(format!( + "# {title}\n\n{}\n\n```policy\n{}\n```\n", + text.trim(), + block.trim_end() + )) +} + /// A module may only add denials: it must be in `package amem.rules`, so a record entry /// cannot redefine the built-in decision, the read filter or the gates. #[must_use] @@ -208,7 +321,9 @@ pub struct Finding { /// The policy id the finding is about. pub policy: String, /// What kind: `implements-unknown`, `higher-layer-conflict`, `exception-on-core`, - /// `exception-expired`, `inert`, `module-refused`. + /// `exception-expired`, `inert`, `module-refused`, `no-owner`, `review-due`, + /// `held-back`; and from the caller, which has the grants and the decision log, + /// `over-ceiling` and `unused`. pub kind: String, /// One sentence. pub why: String, @@ -249,13 +364,14 @@ pub fn lint(entries: &[Entry], now: time::OffsetDateTime) -> Vec { Some(_) => {} } } + out.extend(lifecycle_findings(e, now)); if let Some(b) = &m.breaches { - if by_id(b).is_some_and(|t| t.meta.layer == 0) { + if by_id(b).is_some_and(|t| t.meta.is_core()) { out.push(Finding { policy: m.id.clone(), kind: "exception-on-core".into(), why: format!( - "{} breaches {b}, an L0 rule, which takes no exception: it is ignored", + "{} breaches {b}, a rule of the L0 core, which takes no exception: it is ignored", m.id ), }); @@ -296,6 +412,62 @@ pub fn lint(entries: &[Entry], now: time::OffsetDateTime) -> Vec { out } +/// What a rule's place in its life asks of it: a rule in a decision has an owner and +/// has been looked at by its review date, and a rule is where its block says it is. +fn lifecycle_findings(e: &Entry, now: time::OffsetDateTime) -> Vec { + let m = &e.meta; + let mut out = Vec::new(); + if m.is_exception() { + return out; + } + let state = e.state(); + let live = matches!(state, Status::Enforced | Status::Advisory); + if live && m.owner.as_deref().is_none_or(|o| o.trim().is_empty()) { + out.push(Finding { + policy: m.id.clone(), + kind: "no-owner".into(), + why: format!( + "{} is {} and names no owner to answer for it", + m.id, + state.as_str() + ), + }); + } + let format = time::macros::format_description!("[year]-[month]-[day]"); + let due = m + .review_by + .as_deref() + .and_then(|d| time::Date::parse(d, format).ok()) + .is_some_and(|d| d < now.date()); + if live && due { + out.push(Finding { + policy: m.id.clone(), + kind: "review-due".into(), + why: format!( + "{} was to be reviewed by {}: its owner confirms it, amends it or retires it", + m.id, + m.review_by.as_deref().unwrap_or_default() + ), + }); + } + if let Some(said) = m.status { + if said != state { + out.push(Finding { + policy: m.id.clone(), + kind: "held-back".into(), + why: format!( + "{} says {} and its entry is {}: it is {} until the entry is reviewed that far", + m.id, + said.as_str(), + e.tier, + state.as_str() + ), + }); + } + } + out +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/server/src/lib.rs b/crates/server/src/lib.rs index df4cba68..6ce51c19 100644 --- a/crates/server/src/lib.rs +++ b/crates/server/src/lib.rs @@ -44,6 +44,7 @@ pub mod prefetch; pub mod projection; pub mod reads; pub mod recall; +pub mod rules; pub mod sources; pub mod t0; pub mod tape; @@ -233,7 +234,9 @@ fn policy_entries(ix: &Index, root: &Path, grants: &Grants) -> Vec ", enforced", + (dm_policy::record::Status::Advisory, true) => ", advisory", + _ => "", + }; + let live = matches!( + state, + dm_policy::record::Status::Enforced | dm_policy::record::Status::Advisory + ); + if live && (e.tier == TrustTier::Verified || !how.is_empty()) { + out.push(format!("{} (L{}{how}): {}", m.id, m.layer, m.rule)); + } } } out @@ -6383,6 +6403,254 @@ mod tests { an_exception_lets_one_task_through(dir.path(), &owner, &agent, &d2).await; } + /// A rule whose block says enforced, with a module that gives a code and a remedy. + fn rule_text(id: &str, layer: u8, extra: &str) -> String { + format!( + "# A rule\n\n```policy\nid = \"{id}\"\nlayer = {layer}\nrule = \"Decisions are superseded by a person, not an agent.\"\nintent = \"An agent rewrote a standing decision mid-task.\"\nowner = \"human:test:lead\"\nstatus = \"enforced\"\n{extra}```\n\n```rego\npackage amem.rules\nimport rego.v1\ndeny contains {{\"policy\": \"{id}\", \"reason\": \"an agent superseded a decision\", \"code\": \"decision-by-agent\", \"remedy\": \"propose the change to the owner of the decision\"}} if {{\n\tinput.principal.class == \"agent\"\n\tinput.write.supersedes_type == \"decision\"\n}}\n```\n" + ) + } + + async fn policy_entry(m: &Memory, body: &str) -> String { + m.write(WriteRequest { + entry_type: EntryType::Policy, + ..req(body) + }) + .await + .unwrap() + .id + } + + /// An agent supersedes the newest decision of the record's one chain. + async fn agent_supersedes(root: &Path, of: &str) -> Result { + let agent = Memory::open(root).unwrap().with_author("agent:a"); + agent + .write(WriteRequest { + entry_type: EntryType::Decision, + supersedes: Some(of.to_string()), + ..req("Use something else on a laptop.\n") + }) + .await + } + + /// Where a rule is, to a session opened now. + fn status_of(root: &Path, id: &str) -> Option { + let m = Memory::open(root).unwrap(); + m.check_policy("supersede a decision", None) + .unwrap() + .into_iter() + .find(|r| r.policy_id == id) + .map(|r| r.status) + } + + /// A rule goes up one reviewed step at a time, and is in a decision only once a + /// person verified the version that says so. Advisory reports and refuses nothing; + /// enforced refuses, and says what would pass. + #[tokio::test] + async fn a_rule_is_proposed_then_advisory_then_enforced() { + use dm_policy::record::Status as Rule; + let dir = tempfile::tempdir().unwrap(); + let owner = mem(dir.path()); + let d1 = owner + .write(WriteRequest { + entry_type: EntryType::Decision, + ..req("Use SQLite on a laptop.\n") + }) + .await + .unwrap() + .id; + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + let advisory = rule_text("pol-L2-050", 2, "").replace("\"enforced\"", "\"advisory\""); + let rule = policy_entry(&agent, &advisory).await; + // a curator's check puts no module in a decision: the rule is still proposed + let curator = Memory::open(dir.path()) + .unwrap() + .with_author("agent:curator"); + curator + .promote(&rule, TrustTier::Curated, "checked") + .await + .unwrap(); + assert_eq!(status_of(dir.path(), "pol-L2-050"), None); + let d2 = agent_supersedes(dir.path(), &d1).await.unwrap().id; + let log = || std::fs::read_to_string(dir.path().join("state/decisions.jsonl")).unwrap(); + assert!(!log().contains("pol-L2-050"), "in no decision yet"); + // a person verifies it: advisory, which reports and refuses nothing + verify(&owner, &curator, &rule).await; + assert_eq!(status_of(dir.path(), "pol-L2-050"), Some(Rule::Advisory)); + let d3 = agent_supersedes(dir.path(), &d2).await.unwrap().id; + let all = log(); + let reported = all.lines().last().unwrap_or_default(); + assert!( + reported.contains("\"allow\":true") + && reported.contains("\"advisories\":[{") + && reported.contains("pol-L2-050"), + "{reported}" + ); + // the step up is a new version, which a person verifies: enforced + let owner = Memory::open(dir.path()).unwrap(); + let enforced = owner + .write(WriteRequest { + entry_type: EntryType::Policy, + supersedes: Some(rule.clone()), + ..req(&rule_text("pol-L2-050", 2, "")) + }) + .await + .unwrap() + .id; + assert_eq!(status_of(dir.path(), "pol-L2-050"), None, "under review"); + verify(&owner, &curator, &enforced).await; + assert_eq!(status_of(dir.path(), "pol-L2-050"), Some(Rule::Enforced)); + let refused = agent_supersedes(dir.path(), &d3) + .await + .unwrap_err() + .to_string(); + for part in [ + "pol-L2-050", + "(code decision-by-agent)", + "To pass: propose the change to the owner of the decision.", + ] { + assert!(refused.contains(part), "{part}: {refused}"); + } + } + + /// An exception is asked for one task until a time, is nothing until a person + /// approves it, and is refused for the L0 core. + #[tokio::test] + async fn an_exception_is_filed_for_a_task_and_waits_for_its_approver() { + let dir = tempfile::tempdir().unwrap(); + let owner = mem(dir.path()); + let d1 = owner + .write(WriteRequest { + entry_type: EntryType::Decision, + ..req("Use SQLite on a laptop.\n") + }) + .await + .unwrap() + .id; + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + let core = "# A value\n\n```policy\nid = \"pol-L0-001\"\nlayer = 0\nrule = \"The record is never rewritten in place.\"\nintent = \"History is what an audit reads.\"\n```\n"; + for body in [rule_text("pol-L2-051", 2, ""), core.to_string()] { + let rule = policy_entry(&owner, &body).await; + verify(&owner, &agent, &rule).await; + } + let held = owner + .file_own(tspec("Replace the decision", 0, None, &[], &[])) + .await + .unwrap() + .task; + let ask = |breaches: &str, expires: &str| rules::ExceptionRequest { + breaches: breaches.into(), + task: Some(held.id.clone()), + scope: "this one replacement".into(), + expires: expires.into(), + justification: "The task is to replace the decision.".into(), + domain: None, + }; + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + let core = agent + .file_exception(ask("pol-L0-001", "2099-01-01T00:00:00Z")) + .await; + assert!(matches!(&core, Err(Error::Denied(why)) if why.contains("L0 core"))); + let past = agent + .file_exception(ask("pol-L2-051", "2020-01-01T00:00:00Z")) + .await; + assert!(matches!(&past, Err(Error::Denied(why)) if why.contains("time to come"))); + let unknown = agent + .file_exception(ask("pol-L2-999", "2099-01-01T00:00:00Z")) + .await; + assert!(matches!(unknown, Err(Error::NotFound(_)))); + let filed = agent + .file_exception(ask("pol-L2-051", "2099-01-01T00:00:00Z")) + .await + .unwrap(); + assert_eq!(filed.approver, "human:test:lead", "the owner of an L2 rule"); + assert!(filed.policy_id.starts_with("exc-")); + // filed is not approved: the task is still refused + let inside = Memory::open(dir.path()).unwrap().with_author("agent:a"); + inside.set_held_task(&held.id); + let supersede = || WriteRequest { + entry_type: EntryType::Decision, + supersedes: Some(d1.clone()), + ..req("Use files on a laptop.\n") + }; + assert!(inside.write(supersede()).await.is_err()); + owner.approve(&filed.task.id).await.unwrap(); + let inside = Memory::open(dir.path()).unwrap().with_author("agent:a"); + inside.set_held_task(&held.id); + inside.write(supersede()).await.unwrap(); + } + + /// An amendment is a task for a person that names the rule's owner; the same + /// evidence twice is one task. + #[tokio::test] + async fn an_amendment_is_a_task_for_the_rule_s_owner() { + let dir = tempfile::tempdir().unwrap(); + let owner = mem(dir.path()); + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + let rule = policy_entry(&owner, &rule_text("pol-L2-052", 2, "")).await; + verify(&owner, &agent, &rule).await; + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + let evidence = "The rule refused a decision that its own owner asked to replace."; + let first = agent + .propose_amendment("pol-L2-052", evidence) + .await + .unwrap(); + assert_eq!(first.task.spec.origin.as_deref(), Some("amend:pol-L2-052")); + assert_eq!(first.task.spec.requirements, ["human"]); + assert!(first.task.spec.body.contains("human:test:lead")); + assert!(first.task.spec.body.contains(evidence)); + let again = agent + .propose_amendment("pol-L2-052", evidence) + .await + .unwrap(); + assert!(again.deduplicated); + assert_eq!(again.task.id, first.task.id); + let none = agent.propose_amendment("pol-L2-052", " ").await; + assert!(matches!(none, Err(Error::Denied(_)))); + let unknown = agent.propose_amendment("pol-L2-999", evidence).await; + assert!(matches!(unknown, Err(Error::NotFound(_)))); + } + + /// A reset keeps each rule that a person wrote or promoted and retires the rest, + /// and only the owner resets. + #[tokio::test] + async fn a_reset_keeps_the_rules_with_a_person_in_their_lineage() { + let dir = tempfile::tempdir().unwrap(); + let owner = mem(dir.path()); + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + let curator = Memory::open(dir.path()) + .unwrap() + .with_author("agent:curator"); + // written by a person; written by an agent and verified by a person; written by + // an agent and curated by an agent; written by an agent and never reviewed + policy_entry(&owner, &rule_text("pol-L3-001", 3, "")).await; + let verified = policy_entry(&agent, &rule_text("pol-L3-002", 3, "")).await; + verify(&owner, &curator, &verified).await; + let curated = policy_entry(&agent, &rule_text("pol-L3-003", 3, "")).await; + curator + .promote(&curated, TrustTier::Curated, "checked") + .await + .unwrap(); + policy_entry(&agent, &rule_text("pol-L3-004", 3, "")).await; + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + assert!(matches!( + agent.reset_rules(None).await, + Err(Error::Denied(_)) + )); + let owner = Memory::open(dir.path()).unwrap(); + let reset = owner.reset_rules(None).await.unwrap(); + let kept: Vec<_> = reset.kept.iter().map(|k| k.policy_id.as_str()).collect(); + assert_eq!(kept, ["pol-L3-001", "pol-L3-002"]); + assert_eq!(reset.retired, ["pol-L3-003", "pol-L3-004"]); + assert!(reset.kept.iter().all(|k| k.person == "human:test:me")); + assert_eq!(status_of(dir.path(), "pol-L3-003"), None, "retired"); + assert_eq!( + status_of(dir.path(), "pol-L3-002"), + Some(dm_policy::record::Status::Enforced) + ); + let again = owner.reset_rules(None).await.unwrap(); + assert!(again.retired.is_empty(), "{:?}", again.retired); + } + async fn an_exception_lets_one_task_through( root: &Path, owner: &Memory, diff --git a/crates/server/src/mcp.rs b/crates/server/src/mcp.rs index 651f5a63..0b384a3e 100644 --- a/crates/server/src/mcp.rs +++ b/crates/server/src/mcp.rs @@ -405,6 +405,44 @@ pub struct LateHandoffParams { pub handoff: String, } +/// Parameters of `check_policy`. +#[derive(Deserialize, schemars::JsonSchema, Default)] +pub struct CheckPolicyParams { + /// What you are about to do, in a few words. + pub action: String, + /// The domain; the only granted domain when omitted. + #[serde(default)] + pub domain: Option, +} + +/// Parameters of `file_exception`. +#[derive(Deserialize, schemars::JsonSchema, Default)] +pub struct FileExceptionParams { + /// The policy id of the rule to breach. + pub breaches: String, + /// The one task the exception holds for. Optional in a session that runs on a task. + #[serde(default)] + pub task: Option, + /// What the exception covers, in words. + pub scope: String, + /// When it lapses, RFC 3339. + pub expiry: String, + /// Why it is needed. + pub justification: String, + /// The domain to write it in; the only granted domain when omitted. + #[serde(default)] + pub domain: Option, +} + +/// Parameters of `propose_amendment`. +#[derive(Deserialize, schemars::JsonSchema, Default)] +pub struct ProposeAmendmentParams { + /// The policy id of the rule. + pub policy_id: String, + /// What happened, and the ids of the entries that show it. + pub evidence: String, +} + /// Parameters of `claim`. #[derive(Deserialize, schemars::JsonSchema, Default)] pub struct LeaseParams { @@ -978,6 +1016,60 @@ impl McpServer { ok(json!({ "task": t })) } + #[tool( + description = "The rules in force for what you are about to do: each with its id, layer, rule, intent and whether it is enforced or advisory. Call it before an action that cannot be taken back." + )] + async fn check_policy( + &self, + p: Parameters, + ) -> Result { + let p = p.0; + let rules = self + .memory + .check_policy(&p.action, p.domain.as_deref()) + .map_err(mcp_err)?; + ok(json!({ "rules": rules })) + } + + #[tool( + description = "Ask for an exception to a rule that refused you: for one task, until an expiry, with a justification. It lets nothing through until a person approves it. A rule of the L0 core takes no exception." + )] + async fn file_exception( + &self, + p: Parameters, + ) -> Result { + let p = p.0; + let filed = self + .memory + .file_exception(crate::rules::ExceptionRequest { + breaches: p.breaches, + task: p.task, + scope: p.scope, + expires: p.expiry, + justification: p.justification, + domain: p.domain, + }) + .await + .map_err(mcp_err)?; + ok(json!({ "exception": filed })) + } + + #[tool( + description = "Propose an amendment to a rule you find wrong in the case before you. It files a task for the rule's owner with your evidence. The rule stays in force until its owner changes it." + )] + async fn propose_amendment( + &self, + p: Parameters, + ) -> Result { + let p = p.0; + let created = self + .memory + .propose_amendment(&p.policy_id, &p.evidence) + .await + .map_err(mcp_err)?; + ok(json!({ "task": created.task, "deduplicated": created.deduplicated })) + } + #[tool( description = "Close a task without doing it: obsolete, invalid, or a duplicate of another." )] @@ -1405,6 +1497,71 @@ mod tests { assert_eq!(filed["not_admitted_because"], "it names no task definition"); } + /// The three policy tools of the paper's surface answer over MCP. + #[tokio::test] + async fn the_policy_tools_give_the_rules_an_exception_and_an_amendment() { + let (_dir, s) = server().await; + let rule = "# A rule\n\n```policy\nid = \"pol-L2-060\"\nlayer = 2\nrule = \"A release is tagged by a person.\"\nintent = \"A tag cannot be taken back.\"\nowner = \"human:test:lead\"\nstatus = \"advisory\"\n```\n"; + let written = s + .memory + .write(crate::WriteRequest { + entry_type: dm_core::EntryType::Policy, + body: rule.into(), + ..crate::WriteRequest::default() + }) + .await + .unwrap(); + let agent = crate::Memory::open(s.memory.root()) + .unwrap() + .with_author("agent:curator"); + let crate::Promotion::Proposed { task } = agent + .promote(&written.id, dm_core::TrustTier::Verified, "checked") + .await + .unwrap() + else { + panic!("proposed") + }; + s.memory.approve(&task.id).await.unwrap(); + let rules = payload( + &s.check_policy(Parameters(CheckPolicyParams { + action: "tag a release".into(), + domain: None, + })) + .await + .unwrap(), + ); + assert_eq!(rules["rules"][0]["policy_id"], "pol-L2-060"); + assert_eq!(rules["rules"][0]["status"], "advisory"); + assert_eq!(rules["rules"][0]["intent"], "A tag cannot be taken back."); + let amend = |evidence: &str| ProposeAmendmentParams { + policy_id: "pol-L2-060".into(), + evidence: evidence.into(), + }; + let task = payload( + &s.propose_amendment(Parameters(amend("The tag was a test tag."))) + .await + .unwrap(), + ); + assert_eq!(task["task"]["origin"], "amend:pol-L2-060"); + let id = task["task"]["id"].as_str().unwrap().to_string(); + let filed = payload( + &s.file_exception(Parameters(FileExceptionParams { + breaches: "pol-L2-060".into(), + task: Some(id), + scope: "one test tag".into(), + expiry: "2099-01-01T00:00:00Z".into(), + justification: "The tag is on a scratch repository.".into(), + domain: None, + })) + .await + .unwrap(), + ); + assert_eq!(filed["exception"]["approver"], "human:test:lead"); + assert_eq!(filed["exception"]["task"]["requirements"][0], "human"); + let none = s.propose_amendment(Parameters(amend(""))).await; + assert!(none.is_err()); + } + #[test] fn write_refuses_a_parameter_it_does_not_have() { let old = diff --git a/crates/server/src/rules.rs b/crates/server/src/rules.rs new file mode 100644 index 00000000..0fdef040 --- /dev/null +++ b/crates/server/src/rules.rs @@ -0,0 +1,362 @@ +//! The rule lifecycle as a session meets it: the rules in force where it is about to +//! act, an exception asked for, an amendment proposed to a rule's owner, and the reset +//! by which a domain's owner retires what no person stands behind. +//! +//! A rule is a policy entry ([`dm_policy::record`]). Where it is in its life is its +//! block's `status` held back by its entry's tier, so every step up is a reviewed +//! promotion and every step is a version of an entry. Nothing here is a second store +//! of rules. + +use crate::{Error, Memory, WriteRequest}; +use dm_core::{EntryType, TrustTier}; +use dm_ledger::{Task, TaskSpec}; +use dm_policy::decide::Class; +use dm_policy::record::{self, Meta, Status}; +use serde::{Deserialize, Serialize}; +use time::OffsetDateTime; + +/// The origin of a task that proposes an amendment to a rule, before the rule's id. +pub const AMENDMENT_ORIGIN: &str = "amend:"; + +/// One rule in force, as [`Memory::check_policy`] gives it. +#[derive(Clone, Debug, PartialEq, Eq, Serialize)] +pub struct RuleInForce { + /// The rule's stable id. + pub policy_id: String, + /// 0 values, 1 strategy, 2 practice, 3 domain convention. + pub layer: u8, + /// The rule, one sentence. + pub rule: String, + /// Why: what an edge case is resolved from. + pub intent: String, + /// `enforced`, which refuses, or `advisory`, which reports. + pub status: Status, + /// Whether a module checks it. A rule without one binds by being read. + pub checked: bool, + /// Who answers for it. + #[serde(skip_serializing_if = "Option::is_none")] + pub owner: Option, +} + +/// What a session asks an exception for. +#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct ExceptionRequest { + /// The rule to breach, by its policy id. + pub breaches: String, + /// The one task the exception holds for; the task the session runs on when omitted. + #[serde(default)] + pub task: Option, + /// What it covers, in words. + pub scope: String, + /// When it lapses, RFC 3339. + pub expires: String, + /// Why it is needed. + pub justification: String, + /// The domain to write it in; the only granted domain when omitted. + #[serde(default)] + pub domain: Option, +} + +/// An exception as filed: written, in force for nothing until a person approves it. +#[derive(Clone, Debug, Serialize)] +pub struct FiledException { + /// The entry that holds it. + pub id: String, + /// Its policy id, which a decision that uses it cites. + pub policy_id: String, + /// Who approves: the higher the layer breached, the more senior. + pub approver: String, + /// The approval task. + pub task: Box, +} + +/// What a reset did. +#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize)] +pub struct Reset { + /// The domain. + pub domain: String, + /// The rules kept, each with the person in its lineage. + pub kept: Vec, + /// The rules retired, by policy id. + pub retired: Vec, +} + +/// A rule a reset kept. +#[derive(Clone, Debug, PartialEq, Eq, Serialize)] +pub struct Kept { + /// The rule. + pub policy_id: String, + /// The person who wrote it or promoted it. + pub person: String, +} + +fn is_person(author: &str, owner: &str) -> bool { + matches!(Class::of(author, owner), Class::Owner | Class::Human) +} + +impl Memory { + /// The rules in force for an action in a domain: every enforced and advisory rule + /// of layers 0 to 2, and the domain's own conventions. A session asks before an + /// action it cannot take back. The rules are not chosen by the action: the action + /// is what the read log keeps, so an audit finds that the session asked. + /// + /// # Errors + /// + /// A domain outside the grants. + pub fn check_policy( + &self, + action: &str, + domain: Option<&str>, + ) -> Result, Error> { + let domain = self.resolve_domain(domain)?; + let p = self.policy(); + let mut rules: Vec = p + .entries() + .iter() + .filter(|e| !e.meta.is_exception()) + .filter(|e| e.meta.layer < 3 || e.domain == domain) + .filter_map(|e| { + let status = p.state_of(e); + matches!(status, Status::Enforced | Status::Advisory).then(|| RuleInForce { + policy_id: e.meta.id.clone(), + layer: e.meta.layer, + rule: e.meta.rule.clone(), + intent: e.meta.intent.clone(), + status, + checked: e.rego.is_some(), + owner: e.meta.owner.clone(), + }) + }) + .collect(); + rules.sort_by(|a, b| (a.layer, &a.policy_id).cmp(&(b.layer, &b.policy_id))); + rules.dedup_by(|a, b| a.policy_id == b.policy_id); + self.reads + .text("check_policy", &format!("{action} in {domain}")); + Ok(rules) + } + + /// The rule of a policy id, among the record's rules. + fn rule(&self, policy_id: &str) -> Result { + self.policy() + .entries() + .iter() + .find(|e| e.meta.id == policy_id && !e.meta.is_exception()) + .cloned() + .ok_or_else(|| Error::NotFound(format!("policy {policy_id}"))) + } + + /// Who approves an exception to a rule: the rule's owner for a domain convention or + /// a practice, and the owner of the record for strategy and values. + fn approver_of(&self, rule: &record::Entry) -> String { + match (&rule.meta.owner, rule.meta.layer) { + (Some(owner), 2..) => owner.clone(), + _ => self.grants.owner.clone(), + } + } + + /// Ask for an exception to a rule: for one task, until a time, with a reason. The + /// exception is written as a policy entry and proposed for approval; it lets + /// nothing through until a person approves it, and after its expiry it lets + /// nothing through again. + /// + /// # Errors + /// + /// [`Error::NotFound`] for a rule or a task that is not there; [`Error::Denied`] + /// for a rule of the L0 core, an expiry that is not a time to come, or no task to + /// scope it to; and as [`Memory::write`] and [`Memory::promote`]. + pub async fn file_exception(&self, ask: ExceptionRequest) -> Result { + let rule = self.rule(&ask.breaches)?; + if rule.meta.is_core() { + return Err(Error::Denied(format!( + "{} is a rule of the L0 core, which takes no exception", + rule.meta.id + ))); + } + let now = OffsetDateTime::now_utc(); + let rfc3339 = &time::format_description::well_known::Rfc3339; + if !OffsetDateTime::parse(&ask.expires, rfc3339).is_ok_and(|t| t > now) { + return Err(Error::Denied(format!( + "an exception expires at a time to come, as RFC 3339: {} is not one", + ask.expires + ))); + } + let Some(task) = ask.task.clone().or_else(|| self.held_task()) else { + return Err(Error::Denied( + "an exception holds for one task: name the task".into(), + )); + }; + self.get_task(&task).await?; + let approver = self.approver_of(&rule); + let key = dm_core::digest(&format!("{}\n{task}\n{}", rule.meta.id, ask.expires)); + let hex = key.rsplit(':').next().unwrap_or_default(); + let policy_id = format!("exc-{}-{}", now.year(), hex.get(..8).unwrap_or(hex)); + let meta = Meta { + id: policy_id.clone(), + layer: rule.meta.layer, + intent: ask.justification.clone(), + breaches: Some(rule.meta.id.clone()), + scope_task: Some(task.clone()), + scope: Some(ask.scope.clone()), + expires: Some(ask.expires.clone()), + approver: Some(approver.clone()), + justification: Some(ask.justification.clone()), + ..Meta::default() + }; + let title = format!("Exception to {} for {task}", rule.meta.id); + let body = record::render(&title, &ask.justification, &meta).map_err(Error::Denied)?; + let written = self + .write(WriteRequest { + domain: ask.domain.clone(), + entry_type: EntryType::Policy, + title: Some(title), + body, + cites: vec![rule.entry_id.clone()], + ..WriteRequest::default() + }) + .await?; + let why = format!( + "{approver} approves an exception to {}: {}", + rule.meta.id, ask.justification + ); + match self.promote(&written.id, TrustTier::Verified, &why).await? { + crate::Promotion::Proposed { task } => Ok(FiledException { + id: written.id, + policy_id, + approver, + task, + }), + crate::Promotion::Promoted { .. } => Err(Error::Denied(format!( + "{} was verified without a proposal: an exception takes an approver", + written.id + ))), + } + } + + /// Propose an amendment to a rule: an ordinary task for a person, which names the + /// rule's owner and carries the evidence. The same evidence against the same rule + /// is one task. + /// + /// # Errors + /// + /// [`Error::NotFound`] for a rule that is not there; [`Error::Denied`] for no + /// evidence; a ledger error, or a filing the policy refuses. + pub async fn propose_amendment( + &self, + policy_id: &str, + evidence: &str, + ) -> Result { + let rule = self.rule(policy_id)?; + if evidence.trim().is_empty() { + return Err(Error::Denied( + "an amendment carries its evidence: what happened, and the entries that show it" + .into(), + )); + } + let owner = rule + .meta + .owner + .clone() + .unwrap_or_else(|| self.grants.owner.clone()); + let domain = if rule.domain.is_empty() { + self.resolve_domain(None)? + } else { + rule.domain.clone() + }; + let body = format!( + "For {owner}, who owns {id}.\n\nThe rule: {rule}\n\nIts intent: {intent}\n\n\ + The evidence:\n\n{evidence}\n\nAmend the rule by superseding its entry `{entry}`, \ + retire it, or close this task with why the rule stands.", + id = rule.meta.id, + rule = rule.meta.rule, + intent = rule.meta.intent, + evidence = evidence.trim(), + entry = rule.entry_id, + ); + self.create_task(TaskSpec { + domain, + title: format!("Amend {}", rule.meta.id), + body, + requirements: vec!["human".into()], + origin: Some(format!("{AMENDMENT_ORIGIN}{}", rule.meta.id)), + upstream_event: Some(dm_core::digest(evidence.trim())), + priority: 3, + ..TaskSpec::default() + }) + .await + } + + /// The person in a rule's lineage: who wrote its entry, who promoted it, or who + /// approved its promotion. + async fn person_behind(&self, rule: &record::Entry) -> Result, Error> { + let owner = &self.grants.owner; + let entry = self.get(&rule.entry_id).await?; + if is_person(&entry.front.author, owner) { + return Ok(Some(entry.front.author)); + } + // a promotion to curated names who made it; a promotion to verified names who + // proposed it, and the approval task it closed names who approved + let l = self.ledger.lock().await; + let promoted = l.promotions(&rule.domain, &rule.entry_id).await?; + let approvals = format!("{}{}", crate::PROPOSAL_ORIGIN, rule.entry_id); + let approved = l.list(&rule.domain, dm_ledger::Status::Done).await?; + let approvers = approved + .into_iter() + .filter(|t| t.spec.origin.as_deref() == Some(approvals.as_str())) + .filter_map(|t| t.holder); + Ok(promoted + .into_iter() + .map(|p| p.spec.by) + .chain(approvers) + .find(|by| is_person(by, owner))) + } + + /// Reset a domain's rules: the owner retires, in one step, every rule that no + /// person wrote or promoted, and keeps the rest. A retired rule is superseded by a + /// version whose block says so, as every change to the record is a version. + /// + /// # Errors + /// + /// [`Error::Denied`] for anyone but the owner of the record, or a domain outside + /// the grants; and as [`Memory::write`]. + pub async fn reset_rules(&self, domain: Option<&str>) -> Result { + if self.author != self.grants.owner { + return Err(Error::Denied(format!( + "only the owner {} resets the rules", + self.grants.owner + ))); + } + let domain = self.resolve_domain(domain)?; + let p = self.policy(); + let mut out = Reset { + domain: domain.clone(), + ..Reset::default() + }; + for rule in p.entries().iter().filter(|e| { + e.domain == domain && !e.meta.is_exception() && e.state() != Status::Retired + }) { + if let Some(person) = self.person_behind(rule).await? { + out.kept.push(Kept { + policy_id: rule.meta.id.clone(), + person, + }); + continue; + } + let old = self.get(&rule.entry_id).await?; + let body = record::with_status(&old.body, Status::Retired).map_err(Error::Denied)?; + self.write(WriteRequest { + domain: Some(domain.clone()), + entry_type: EntryType::Policy, + title: old.front.title.clone(), + body, + supersedes: Some(rule.entry_id.clone()), + ..WriteRequest::default() + }) + .await?; + out.retired.push(rule.meta.id.clone()); + } + out.kept.sort_by(|a, b| a.policy_id.cmp(&b.policy_id)); + out.retired.sort(); + self.reload_policy().await?; + Ok(out) + } +} diff --git a/docs/demo-narrative.md b/docs/demo-narrative.md index c4411941..dea0d3ba 100644 --- a/docs/demo-narrative.md +++ b/docs/demo-narrative.md @@ -76,7 +76,7 @@ offline. - **Egress**: the only way out to a model. Every model call goes through it, from a harness or from the system itself, on to the model's provider or the gateway. The egress checks, logs and meters each call. Web fetches do not go through it. They leave - through the memory server's own fetch tool (3.9). + through the memory server's own fetch tool (3.11). - **Harness**: the agent program that a model works in, such as Claude Code or Codex. - **Gateway**: llm.apache.org, the Foundation's LLM gateway. It serves the open models in this demo. @@ -90,7 +90,7 @@ offline. (act 8). - **Tap**: a component that connects to an outside system. It runs as a WebAssembly component, and it can reach only what its grant names. The fetch tool in - 3.9 is a tap. The GitHub tap in act 9 is another. + 3.11 is a tap. The GitHub tap in act 9 is another. - **Event log**: every write and erasure in order, by id and digest, never a body. The index can be rebuilt from the event log and the record (act 10). - **Bundle**: entries exported at or below a label, as Markdown and a manifest, for @@ -185,24 +185,42 @@ it, and what did they write after? Here `agent:reader` read one and wrote a note the list shows the note. **3.5 Policy is memory.** A rule written as an entry: a person, not an agent, supersedes -decisions. While the rule is raw, it does nothing, and `agent:a` supersedes a decision. -After you verify the rule, the policy holds the same kind of write. +decisions. The rule has an owner and a review date. Its block says `advisory`, as for a +rule that someone writes after an incident. The rule goes up one step at each review: -**3.6 An exception.** You approve an exception for one task, until a set date. Every -session sees it in T0. +- While the rule is raw, it is proposed. It does nothing, and `agent:a` supersedes a + decision. +- After you verify the rule, it is advisory. `agent:a` supersedes a decision again. The + decision log says that the rule would refuse that write. +- You write the version of the rule that says `enforced`. Until you verify that version, + the rule is proposed again. +- After you verify that version, the rule is enforced. The policy holds the same kind of + write. The refusal gives the rule, a reason code and what would pass. -**3.7 The lint.** This shows how often each rule blocked or allowed something. +**3.6 Before an action.** `agent:a` asks which rules are in force, and gets each rule +with its intent. Then `agent:a` proposes an amendment to the rule. The amendment is a +task for a person, and it names the owner of the rule. -**3.8 Spend limits.** For this step, the egress gives the auditor a spend limit of 400 +**3.7 An exception.** `agent:a` asks for an exception for one task, until a set time. +The task is still refused. You approve the exception, and then the task goes through. +Every session sees the exception in T0. + +**3.8 A reset.** `agent:a` writes a rule, and `agent:curator` promotes it to curated. No +person wrote or promoted that rule. You reset the rules of the domain. The reset retires that rule. +It keeps the rule that you verified. + +**3.9 The lint.** This shows how often each rule blocked, allowed or advised something. + +**3.10 Spend limits.** For this step, the egress gives the auditor a spend limit of 400 tokens at the gateway. The call that goes over the limit still goes out. The egress refuses the next call. By then, more than 550 tokens are spent. -**3.9 Fetch.** `agent:reader` fetches web pages through memory. No grant names +**3.11 Fetch.** `agent:reader` fetches web pages through memory. No grant names example.com. After `agent:reader` has read internal entries, it can reach only trusted hosts, because a URL could leak what it read. The repeated fetch comes from the stored copy. Nothing leaves. -**3.10 Reads.** A policy limits agents to public entries. When `agent:a` searches, it +**3.12 Reads.** A policy limits agents to public entries. When `agent:a` searches, it finds one hit. Your own search reaches the cap of 50 results. ## Act 4 · The task fabric diff --git a/docs/terms.md b/docs/terms.md index 2e4ad4e7..54a86106 100644 --- a/docs/terms.md +++ b/docs/terms.md @@ -50,6 +50,11 @@ term here in the same commit as the first document that needs it. | grant | One table in the grants: `[domains.]`, `[destinations.]` or `[taps.]` | permission | | policy | The Rego rules that decide each action: write, read, erase, promote, file a task, spend and fetch. The root's `policy/*.rego` replaces the built-in rules | rules engine | | perimeter | The policy, the grants and the parts that enforce them, as one part of the design | | +| rule | One policy entry in the record: an id, a layer, the rule, its intent, an owner and, where a machine can check it, a module | | +| status (of a rule) | Where a rule is in its life: `proposed` (in no decision), `advisory` (reports, refuses nothing), `enforced` (refuses) or `retired` | state (of a rule) | +| ceiling | `[gates] max_enforced_rules`: the largest number of enforced rules in one domain that need no sign-off | cap (for rules) | +| reset | `amem policy reset`: the owner retires each rule of a domain that no person wrote, promoted or approved | purge | +| amendment | A task for the owner of a rule, with evidence, that asks for a change to the rule | | | gates | The `[gates]` rules on a write: how many entries one session may supersede, and whether a verified entry needs an approver | | | scan | The check of a write and of a tool's answer for instructions, secrets, hidden text and the like. The scan finds; the policy decides | | | label | `public`, `internal` or `restricted`, in that order | level, tier (for a label), sensitivity | diff --git a/local/README.md b/local/README.md index 3bd1a41c..8b4032fe 100644 --- a/local/README.md +++ b/local/README.md @@ -524,20 +524,72 @@ body of such an entry has a fenced `policy` block in TOML. The block gives: - the layer, 0 to 3; - the rule and its intent; - what the rule implements; -- who owns the rule. +- who owns the rule (`owner`); +- the status that the author wants (`status`); +- the date of the next review (`review_by`, a calendar date). Where a machine can check the rule, a fenced `rego` block adds denials in -`package amem.rules`. - -`amem` enforces a module only when the entry of the module is verified. Thus an agent -can propose a rule, but the rule has no effect until a person approves it. This is the -paper's evolution-loop rule: nothing below verified loads as live configuration. `amem` -refuses a module outside the rules package, or a module that does not load. It -names that module and leaves it out, and the base policy stays in force. +`package amem.rules`. A denial gives `policy` and `reason`. It can also give `code`, a +short reason code, and `remedy`, what would pass. A refusal shows all of them. + +A rule has one of four statuses: `proposed`, `advisory`, `enforced` or `retired`. The +`status` in the block is the highest status that the rule can have. The trust tier of +the entry holds the rule back: + +| `status` in the block | raw or curated | verified | +| --------------------- | -------------- | -------- | +| `enforced`, or no status | proposed | enforced | +| `advisory` | proposed | advisory | +| `proposed` | proposed | proposed | +| `retired` | retired | retired | + +Thus an agent can propose a rule, but the rule is in no decision until a person approves +it. `amem` loads no module from an entry below verified. A step from advisory to +enforced is a new version of the entry, with `status = "enforced"`. The new version +supersedes the old version, so the rule is proposed until a person verifies the new +version. `amem` evaluates the module of an advisory rule with each decision. It writes what the module would refuse to the decision +log (`advisories`), and it refuses nothing for that rule. `amem` refuses a module outside +the rules package, or a module that does not load. It names that module and leaves it +out, and the base policy stays in force. + +A session loads the policy when it starts, and again when it changes a policy entry +itself. A change that a different session makes applies to the sessions that start after +the change. + +`[gates] max_enforced_rules` in `grants.toml` is a ceiling on the enforced rules of one +domain. There is no ceiling by default. `amem` counts the rules of a domain in the order +of their ids. A rule over the ceiling is advisory. A rule with `signed_off_by` in its +block is enforced and is not counted, so a sign-off admits one rule and does not raise +the ceiling. + +An L0 rule is in the core unless its block says `core = false`. A rule in the core takes +no exception. An exception is a policy entry that names three things: the policy that it breaches, the one task that it holds for, and an expiry. An exception is in force when it is verified -and not expired. An exception is never in force for an L0 rule. +and not expired. An exception is never in force for a rule in the L0 core. + +Three memory tools use the rules. Each tool is also a command: + +- `check_policy` (`amem policy check `) gives the rules in force for a domain: + the enforced rules and the advisory rules of layers 0 to 2, and the domain's own L3 + rules. Each rule has its id, layer, rule, intent and status. `amem` does not select + the rules by the action. The read log records that the session asked. +- `file_exception` (`amem policy except --task --scope --expiry + --justification`) writes an exception and proposes it for verified. The answer names + the approver: the owner of the rule for layers 2 and 3, and the owner of the record + for layers 0 and 1 and for a rule with no owner. `amem approve` on the task of the answer puts the exception in + force. `amem` does not check that the person who approves is the approver that the + answer names. +- `propose_amendment` (`amem policy amend --evidence`) files a task for a + person. The task names the owner of the rule and carries the evidence. The same + evidence for the same rule is one task. + +`amem policy reset` retires the rules of a domain that have no person in their lineage. +Only the owner of the record can run it. A rule has a person in its lineage when a +person wrote its entry, promoted it, or approved its promotion. The reset keeps those +rules. For each other rule, the reset writes a new version with `status = "retired"` +that supersedes the old version. The reset does not change exceptions. A denial from a record rule wins over the built-in decision, unless an exception covers the task that the session holds. Each decision names the policies that it rests on: the @@ -549,11 +601,21 @@ verified rules by id, and the exceptions in force. lint` applies the paper's conflict rules. It finds: - a lower layer that implements what is not there or a higher layer; -- an exception against L0; +- an exception against the L0 core; - an expired exception; -- a rule that is not yet verified. - -With `--file`, `amem policy lint` files a bug on the lower layer for the owner. +- a rule that is not yet verified; +- an enforced or advisory rule with no owner (`no-owner`); +- an enforced or advisory rule after its `review_by` date (`review-due`); +- a rule whose status is lower than its block says (`held-back`); +- a rule that the ceiling holds at advisory (`over-ceiling`); +- an enforced rule that no decision in the decision log cites (`unused`). + +It also gives, for each rule, how often it refused, how often an exception let a write +through, and how often the rule advised. + +With `--file`, `amem policy lint` files a task for the owner for each finding, except +for a rule that is not yet verified. Thus an unused rule goes to the owner. `amem` looks +at the full decision log, not at one review period. ## The states of a task, and its claim: `amem task` diff --git a/scripts/demo/3-perimeter.sh b/scripts/demo/3-perimeter.sh index f541fecf..acd303da 100755 --- a/scripts/demo/3-perimeter.sh +++ b/scripts/demo/3-perimeter.sh @@ -27,60 +27,77 @@ read_one=$(a list --kind decision --limit 3 | j "d[-1]['id']") agent agent:reader batch "[{\"tool\": \"get\", \"arguments\": {\"id\": \"$read_one\"}}, {\"tool\": \"write\", \"arguments\": {\"title\": \"Drawn from what I read\", \"body\": \"A conclusion drawn from the decision just read.\"}}]" >/dev/null a taint "$read_one" | j "f\" {d['id']}: read by {len(d['readers'])} session(s), who wrote {d['written']} after reading it\"" -say "Policy is memory" "A rule written as an entry, with a Rego module. While it is raw it is inert; once an agent proposes it and the owner approves it for verified, it is enforced, and a write it forbids is held." -rule=$(a write --kind policy --title "pol-L2-031 Decisions are superseded by a person" <<'POL' | j "d['id']" +say "Policy is memory" "A rule is an entry, with an owner, a review date and a Rego module. A rule written after an incident enters as advisory. It is in a decision only once a person verified it, and each step up is a version that a person verifies." +rule_text() { cat </dev/null; } +rules() { a policy list | j "'\n'.join(f\" {p['id']} L{p['layer']} {p['tier']:<8} {p['status']:<9} {p['rule']}\" for p in d['policies'])"; } +rule=$(rule_text advisory | a write --kind policy --title "pol-L2-031 Decisions are superseded by a person" | j "d['id']") +rules d1=$(a write --kind decision --title "Laptops run SQLite" --body "The ledger runs on SQLite on a laptop." | j "d['id']") -echo " while it is raw, an agent may still supersede a decision:" +echo " while it is proposed, it is in no decision, and an agent supersedes a decision:" d2=$(agent agent:a write "{\"title\": \"Laptops run Postgres\", \"body\": \"The ledger runs on Postgres.\", \"entry_type\": \"decision\", \"supersedes\": \"$d1\"}" | j "d['id']") echo " $d2 replaced $d1" -proposal=$(agent agent:curator promote "{\"id\": \"$rule\", \"to\": \"verified\", \"why\": \"the incident record supports it\"}" | j "d['task']['id']") -a approve "$proposal" >/dev/null -echo " an agent proposed it for verified and the owner approved; now it is enforced:" -a policy list | j "'\n'.join(f\" {p['id']} {p['tier']} {'enforced' if p['enforced'] else 'inert'}\" for p in d['policies'])" -agent agent:a write "{\"title\": \"Laptops run files\", \"body\": \"Plain files.\", \"entry_type\": \"decision\", \"supersedes\": \"$d2\"}" | sed 's/^/ /' || true +verified "$rule" "the incident record supports it" +echo " an agent proposed it for verified and the owner approved; now it is advisory. It reports, and it refuses nothing:" +rules +d3=$(agent agent:a write "{\"title\": \"Laptops run DuckDB\", \"body\": \"The ledger runs on DuckDB.\", \"entry_type\": \"decision\", \"supersedes\": \"$d2\"}" | j "d['id']") +grep '"action":"write"' "$root/state/decisions.jsonl" | tail -1 | j "f\" the decision log: write allowed; would refuse: {', '.join(x['policy'] + ' (' + x.get('code', '') + ')' for x in d['advisories'])}\"" +rule=$(rule_text enforced | a write --kind policy --title "pol-L2-031 Decisions are superseded by a person" --supersedes "$rule" | j "d['id']") +echo " the owner writes the version that says enforced; until a person verifies that version, it is proposed:" +rules +verified "$rule" "one month as advisory, and each report was a real case" +echo " verified; now it is enforced:" +rules +echo " the refusal names the rule, a reason code and what would pass:" +agent agent:a write "{\"title\": \"Laptops run files\", \"body\": \"Plain files.\", \"entry_type\": \"decision\", \"supersedes\": \"$d3\"}" | sed 's/^/ /' || true -say "An exception" "A rule can be excepted for one task, until a date, and only once a person approves the exception. Every session loads the policy digest with T0, so the exception is visible, not hidden." -held=$(a task create "Replace the laptop decision" --domain dom-demo --body "The task is to replace it." | j "d['id']") -exc=$(a write --kind policy --title "exc-2026-001 The laptop decision task" </dev/null -echo " holding $held, the agent may:" -agent agent:a write "{\"title\": \"Laptops run files\", \"body\": \"Plain files.\", \"entry_type\": \"decision\", \"supersedes\": \"$d2\"}" "$held" | j "f\" wrote {d['id']}\"" || true +say "An exception" "A rule can be excepted for one task, until an expiry, and only once a person approves the exception. The agent asks; the approver is the rule's owner. Every session loads the policy digest with T0, so the exception is visible, not hidden." +held=$(a task create "Replace the laptop decision" --domain dom-demo --body "The task is to replace it." | j "d['id']") +exception=$(agent agent:a file_exception "{\"breaches\": \"pol-L2-031\", \"task\": \"$held\", \"scope\": \"the one replacement this task exists for\", \"expiry\": \"2027-12-31T00:00:00Z\", \"justification\": \"The task exists to replace that decision.\"}") +printf '%s' "$exception" | j "f\" {d['exception']['policy_id']} filed, to be approved by {d['exception']['approver']}\"" +echo " until a person approves it, the task is still refused:" +agent agent:a write "{\"title\": \"Laptops run files\", \"body\": \"Plain files.\", \"entry_type\": \"decision\", \"supersedes\": \"$d3\"}" "$held" | sed 's/^/ /' | cut -c1-160 || true +a approve "$(printf '%s' "$exception" | j "d['exception']['task']['id']")" >/dev/null +echo " approved; holding $held, the agent may:" +agent agent:a write "{\"title\": \"Laptops run files\", \"body\": \"Plain files.\", \"entry_type\": \"decision\", \"supersedes\": \"$d3\"}" "$held" | j "f\" wrote {d['id']}\"" || true echo " the policy digest every session loads first, in T0:" a t0 | sed -n '/## Policy digest/,/## Pointers/p' | sed '$d' | sed 's/^/ /' -say "The lint" "Each rule and exception with how often it refused and let through, counted from the decision log, so a rule nobody hits or an exception everybody uses shows up." -a policy lint | j "'\n'.join([f\" {f['policy']}: {f['kind']}\" for f in d['findings']] + [f\" {r['id']}: refused {r['refused']}, let through {r['let_through']}\" for r in d['rates']])" + +say "A reset" "Rules are cheapest to add when something has just gone wrong. When the stack has grown past use, the owner retires in one step every rule that no person wrote or promoted, and keeps the rest." +agent agent:a write "{\"title\": \"pol-L3-900 No writes on a Friday\", \"entry_type\": \"policy\", \"body\": \"# No writes on a Friday\n\n\`\`\`policy\nid = \\\"pol-L3-900\\\"\nlayer = 3\nrule = \\\"No entry is written on a Friday.\\\"\nintent = \\\"A write on a Friday went wrong once.\\\"\nstatus = \\\"enforced\\\"\n\`\`\`\n\"}" >/dev/null +agent agent:curator promote "{\"id\": \"$(a policy list | j "[p['entry'] for p in d['policies'] if p['id'] == 'pol-L3-900'][0]")\", \"to\": \"curated\", \"why\": \"it parses\"}" >/dev/null +rules +a policy reset --domain dom-demo | j "'\n'.join([f\" kept {k['policy_id']}: {k['person']} is in its lineage\" for k in d['kept']] + [f\" retired {r}\" for r in d['retired']])" +rules + +say "The lint" "Each rule and exception with how often it refused, let through and advised, counted from the decision log, so a rule nobody hits or an exception everybody uses shows up. A rule past its review date, a rule without an owner and an enforced rule that no decision cites are findings for the owner." +a policy lint | j "'\n'.join([f\" {f['policy']}: {f['kind']}\" for f in d['findings']] + [f\" {r['id']}: refused {r['refused']}, let through {r['let_through']}, advised {r['advised']}\" for r in d['rates']])" if gateway; then say "Spend limits" "A budget per destination and session in the grants. The egress, the only way out, checks what the session has spent before each model call leaves: the call that crosses the limit still goes, the next one is refused." From 30d47809325c58d97c08e65f8eadb88f28173cd0 Mon Sep 17 00:00:00 2001 From: Sander Striker Date: Thu, 12 Nov 2026 14:36:41 +0100 Subject: [PATCH 2/4] fix: close what the adversarial pass found in the rule lifecycle - An advisory module could refuse. Its denial could name a different rule, and the decision used that name. The modules of advisory rules now run in an engine of their own, and their denials are only reported. - A second entry with the id of an enforced rule and `status = "advisory"` switched that rule off. The id of a denial no longer selects a status. - A denial with a field that is not a string made each decision fail. A denial now reads such a field as its JSON. - The words of `file_exception` could carry a policy block and a module into the entry that a person verifies. A request with a code fence is refused, the entry must read back as the exception that was asked for, and the entry goes in the domain of the rule. - The ceiling counted by id, so a new rule could push out an older rule or a rule of the L0 core. It now counts from the oldest entry, never holds back the core, and accepts a sign-off only from a person. - `check_policy` and `propose_amendment` gave the text of a rule above the handling of the session. A rule is now read as an entry is. - An entry that nobody reviewed could speak for a rule with the same id. The tools now use the entry in force. - The reset kept a rule when a person proposed it and nobody approved. It now counts a curation or an approval. Its answer names entries. - T0 showed an L0 rule that nobody verified as core. - An exception with `status = "retired"` stayed in force. - One filer has one open amendment for one rule. - The lint, the rates and the filing of findings are in the library. The `held-back` finding is gone, because `inert` says the same. Co-Authored-By: Claude Fable 5.2 --- PLAN.md | 2 +- README.md | 5 +- crates/memory/src/main.rs | 134 +----------- crates/policy/rego/amem.rego | 26 +-- crates/policy/src/decide.rs | 371 ++++++++++++++++++++++++++------- crates/policy/src/record.rs | 29 +-- crates/server/src/lib.rs | 274 ++++++++++++++++++++++-- crates/server/src/mcp.rs | 10 +- crates/server/src/rules.rs | 394 ++++++++++++++++++++++++++++------- local/README.md | 59 ++++-- scripts/demo/3-perimeter.sh | 2 +- 11 files changed, 946 insertions(+), 360 deletions(-) diff --git a/PLAN.md b/PLAN.md index 381e8aab..1615628e 100644 --- a/PLAN.md +++ b/PLAN.md @@ -70,7 +70,7 @@ Landed so far: | 1 | #71 | Nothing | | 2 | #73 | An attempt key on every operation; an uncertain external effect blocking reassignment; waiting on an outside event as a state; the record's remaining fields (`parent`, `scores`, `implements`, `reversibility`, `artifacts_touched`, `policy_context`) | | 3 | #74 | A suite that gates a version of a definition; the data scope enforced; a definition required by default, once the system's own filings name one; tools offered a few at boot and the rest by search | -| 6 | this pull request | `check_policy` selecting rules by the action; the unused-rule check over one review period; the approver of an exception checked against the layer breached; the ceiling counting rules by age; a session that reloads the policy when a rule changes; a verified rule that stays in force while its next version waits for review | +| 6 | this pull request | `check_policy` selecting rules by the action; the unused-rule check over one review period; the approver of an exception checked against the layer breached; a new version of a rule keeping its place under the ceiling; a session that reloads the policy when a rule changes; a verified rule that stays in force while its next version waits for review | What stays unbuilt after these, and stays on README's list: federation across domains, authentication of people, logprob-based drift, and accelerator time, which only a gateway can count. So do the items on that list that need another machine or an outside service more than they need code: the limits on re-enactment, harness settings and decomposition templates in the platform domain, bytes stored and resource limits on the sandbox, the authoring of agent definitions, the search tool against its live provider, a harness under Seatbelt on a Mac, and the Gofannon bridge against a live deployment, whose agents make model calls that the egress does not meter. diff --git a/README.md b/README.md index 2a1f2503..335e2b24 100644 --- a/README.md +++ b/README.md @@ -127,8 +127,9 @@ two weeks. It is also the reason that none of it is the production shape. domain. It does not select rules by the action. The lint reports a rule after its review date and an unused rule, from the full decision log and not from one review period. The approver of an exception is a name in the answer. `amem` does not check - that this person approves. The order of rules under the ceiling is the order of their - ids. While the next version of a rule waits for its review, the rule is in no decision. + that this person approves. A new version of a rule is the newest rule under the + ceiling. While the next version of a rule waits for its review, the rule is in no + decision. - **Lineage, in part.** An entry does not record its session. `amem taint` compares the time of a read with the time of a later write, and it goes forward only. Reads and sends are not in the event log. A write that the scan refuses is a file under `state/`, diff --git a/crates/memory/src/main.rs b/crates/memory/src/main.rs index 59800446..d1da10a8 100644 --- a/crates/memory/src/main.rs +++ b/crates/memory/src/main.rs @@ -1074,8 +1074,8 @@ enum PolicyOp { #[arg(long)] domain: Option, }, - /// Ask for an exception to a rule, for one task, until an expiry. A person approves - /// it with `amem approve`. + /// Ask for an exception to a rule, for one task, until an expiry. A person who did + /// not ask for it approves it with `amem approve`. Except { /// The policy id of the rule to breach. breaches: String, @@ -1091,8 +1091,6 @@ enum PolicyOp { /// Why it is needed. #[arg(long)] justification: String, - #[arg(long)] - domain: Option, }, /// Propose an amendment to a rule: a task for the rule's owner, with the evidence. Amend { @@ -4705,87 +4703,12 @@ fn policy_list(m: &Memory) -> serde_json::Value { }) } -/// Citations and overrides per policy, from the decision log: how often each rule -/// refused something, and how often an exception let something through. -fn policy_rates(root: &Path) -> serde_json::Value { - let text = std::fs::read_to_string(root.join("state/decisions.jsonl")).unwrap_or_default(); - let mut cited: std::collections::BTreeMap = - std::collections::BTreeMap::new(); - for line in text.lines() { - let Ok(v) = serde_json::from_str::(line) else { - continue; - }; - let allow = v["allow"].as_bool().unwrap_or(false); - for id in v["cites"] - .as_array() - .into_iter() - .flatten() - .filter_map(|c| c.as_str()) - { - let row = cited.entry(id.to_string()).or_default(); - if allow { - row.1 += 1; - } else { - row.0 += 1; - } - } - // what an advisory rule would have refused, had it been enforced - for id in v["advisories"] - .as_array() - .into_iter() - .flatten() - .filter_map(|a| a["policy"].as_str()) - { - cited.entry(id.to_string()).or_default().2 += 1; - } - } - serde_json::json!(cited - .into_iter() - .map(|(id, (refused, overridden, advised))| serde_json::json!({"id": id, "refused": refused, "let_through": overridden, "advised": advised})) - .collect::>()) -} - -/// The findings that need the grants and the decision log: a rule held at advisory by -/// the ceiling on enforced rules, and an enforced rule that no decision in the log -/// cites, which goes to its owner to retire or to say why it stays. -fn ceiling_and_use(m: &Memory, p: &dm_policy::decide::Policy) -> Vec { - let mut out = Vec::new(); - for id in p.over_ceiling(m.grants()) { - out.push(dm_policy::record::Finding { - why: format!( - "{id} is over the ceiling on enforced rules and is advisory: a person signs this one rule off with signed_off_by, or a rule is retired" - ), - policy: id, - kind: "over-ceiling".into(), - }); - } - let rates = policy_rates(m.root()); - let cited: Vec<&str> = rates - .as_array() - .into_iter() - .flatten() - .filter_map(|r| r["id"].as_str()) - .collect(); - for e in p.entries().iter().filter(|e| e.enforced()) { - if !cited.contains(&e.meta.id.as_str()) { - out.push(dm_policy::record::Finding { - policy: e.meta.id.clone(), - kind: "unused".into(), - why: format!( - "{} is enforced and no decision in the log cites it: its owner retires it, or says why it stays", - e.meta.id - ), - }); - } - } - out -} - /// `amem policy check`, `except`, `amend` and `reset`: the rule lifecycle. async fn rule_cmd(m: &Memory, op: PolicyOp) -> Fallible { match op { PolicyOp::Check { action, domain } => { - json(&serde_json::json!({ "rules": m.check_policy(&action, domain.as_deref())? })) + let rules = m.check_policy(&action, domain.as_deref()).await?; + json(&serde_json::json!({ "rules": rules })) } PolicyOp::Except { breaches, @@ -4793,7 +4716,6 @@ async fn rule_cmd(m: &Memory, op: PolicyOp) -> Fallible { scope, expiry, justification, - domain, } => { let ask = dm_server::rules::ExceptionRequest { breaches, @@ -4801,7 +4723,6 @@ async fn rule_cmd(m: &Memory, op: PolicyOp) -> Fallible { scope, expires: expiry, justification, - domain, }; json(&m.file_exception(ask).await?) } @@ -4823,47 +4744,14 @@ async fn rule_cmd(m: &Memory, op: PolicyOp) -> Fallible { /// `amem policy lint`. async fn policy_lint(m: &Memory, file: bool) -> Fallible { - let p = m.policy(); - let mut findings = dm_policy::record::lint(p.entries(), time::OffsetDateTime::now_utc()); - findings.extend( - p.refused() - .iter() - .map(|(id, why)| dm_policy::record::Finding { - policy: id.clone(), - kind: "module-refused".into(), - why: format!("{id}'s module would not load: {why}"), - }), - ); - findings.extend(ceiling_and_use(m, &p)); - let mut filed = Vec::new(); - if file { - let domain = m.resolve_domain(None)?; - for f in findings.iter().filter(|f| f.kind != "inert") { - let created = m - .file_own(dm_ledger::TaskSpec { - domain: domain.clone(), - title: format!("Policy bug, {}: {}", f.kind, f.policy), - body: format!( - "{}\n\nAcross layers the higher wins and the lower is fixed: amend or supersede {} so it agrees, or file an exception with a scope, an expiry and an approver. Within a layer, resolve from the stated intents and record the trade-off.", - f.why, f.policy - ), - requirements: vec!["human".into()], - depends_on: Vec::new(), - origin: Some(format!("policy:{}", f.kind)), - upstream_event: Some(f.policy.clone()), - created_by: String::new(), - classification: dm_core::Label::Internal, - priority: 4, - ..dm_ledger::TaskSpec::default() - }) - .await?; - if !created.deduplicated { - filed.push(created.task.id); - } - } - } + let findings = m.policy_findings(); + let filed = if file { + m.file_policy_findings(&findings).await? + } else { + Vec::new() + }; json(&serde_json::json!({ - "findings": findings, "rates": policy_rates(m.root()), "filed": filed, + "findings": findings, "rates": m.policy_rates(), "filed": filed, })) } diff --git a/crates/policy/rego/amem.rego b/crates/policy/rego/amem.rego index 061bb836..5373e3ec 100644 --- a/crates/policy/rego/amem.rego +++ b/crates/policy/rego/amem.rego @@ -368,22 +368,17 @@ base_decision := erase_decision if input.action == "erase" # `deny contains {"policy": , "reason": }` for a request. A denial wins over # the decision above unless an exception in force covers it: an exception entry, # verified and unexpired, that breaches that policy and is scoped to the task this -# session holds. An L0 rule takes no exception. Every decision says which policies it -# cites, the denials or the exceptions used: the paper's citation and override rates. +# session holds. A rule of the L0 core takes no exception. Every decision says which +# policies it cites, the denials or the exceptions used: the paper's citation and +# override rates. Only the modules of enforced rules are loaded here. The module of an +# advisory rule is evaluated apart, so nothing it says can refuse, whatever rule it +# names. core_policy(pid) if { some p in data.policies p.id == pid p.core == true } -# An advisory rule is evaluated and reported, and refuses nothing. A rule the data does -# not list, a built-in one, is enforced. -advisory_policy(pid) if { - some p in data.policies - p.id == pid - p.status == "advisory" -} - excepted(pid) if { not core_policy(pid) some e in data.exceptions @@ -399,16 +394,8 @@ all_denials contains d if some d in builtin_denials rule_denials contains d if { some d in all_denials not excepted(d.policy) - not advisory_policy(d.policy) } -advisory_denials contains d if { - some d in all_denials - advisory_policy(d.policy) -} - -advisory_list := [d | some d in advisory_denials] - denial_list := [d | some d in rule_denials] exceptions_used contains x if { @@ -426,7 +413,7 @@ denied_list := [p | some d in rule_denials; p := d.policy] denied_reasons := [r | some d in rule_denials; r := sprintf("%v: %v", [d.policy, d.reason])] -decision := object.union(base_decision, {"cites": used_list, "advisories": advisory_list}) if { +decision := object.union(base_decision, {"cites": used_list}) if { count(rule_denials) == 0 } @@ -437,7 +424,6 @@ decision := { "reason": concat("; ", denied_reasons), "cites": denied_list, "denials": denial_list, - "advisories": advisory_list, } if { count(rule_denials) > 0 } diff --git a/crates/policy/src/decide.rs b/crates/policy/src/decide.rs index 9aab9657..e8683341 100644 --- a/crates/policy/src/decide.rs +++ b/crates/policy/src/decide.rs @@ -258,22 +258,50 @@ pub struct Decision { pub advisories: Vec, } -/// One rule's refusal, as its module states it. +/// One rule's refusal, as its module states it. A module is a person's or an agent's +/// text: a field it leaves out is empty, and one that is not a string is read as its +/// JSON, so no module's denial can make a decision fail to parse. #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] pub struct Denial { /// The rule. + #[serde(default, deserialize_with = "as_text")] pub policy: String, /// Why, in a sentence. - #[serde(default)] + #[serde(default, deserialize_with = "as_text")] pub reason: String, /// A short code for the reason, the same for every refusal of its kind. - #[serde(default, skip_serializing_if = "Option::is_none")] + #[serde( + default, + deserialize_with = "as_some_text", + skip_serializing_if = "Option::is_none" + )] pub code: Option, /// What would pass. - #[serde(default, skip_serializing_if = "Option::is_none")] + #[serde( + default, + deserialize_with = "as_some_text", + skip_serializing_if = "Option::is_none" + )] pub remedy: Option, } +fn text_of(v: serde_json::Value) -> String { + match v { + serde_json::Value::String(s) => s, + serde_json::Value::Null => String::new(), + other => other.to_string(), + } +} + +fn as_text<'de, D: serde::Deserializer<'de>>(d: D) -> Result { + serde_json::Value::deserialize(d).map(text_of) +} + +fn as_some_text<'de, D: serde::Deserializer<'de>>(d: D) -> Result, D::Error> { + let v = serde_json::Value::deserialize(d)?; + Ok((!v.is_null()).then(|| text_of(v))) +} + impl Denial { /// The refusal in a sentence a caller can act on: the rule, why, the code, and what /// would pass. @@ -290,29 +318,35 @@ impl Denial { } } -/// The enforced rules over a ceiling, which is a domain's: in each domain, counted in -/// the order of their ids, each one past the ceiling that carries no sign-off of its -/// own. A signed-off rule is enforced and takes no place under the ceiling. -fn over_ceiling(entries: &[crate::record::Entry], ceiling: Option) -> Vec { - let Some(ceiling) = ceiling else { +/// The entries of the enforced rules over a ceiling, which is a domain's. In each +/// domain the enforced rules are counted oldest first, and each one past the ceiling is +/// held back. A rule of the L0 core is never held back and takes no place. Nor does a +/// rule a person signed off: a signature admits that rule and does not raise the +/// ceiling. A new rule can thus never push an older one out. +fn over_ceiling(entries: &[crate::record::Entry], grants: &Grants) -> Vec { + let Some(ceiling) = grants.gates.max_enforced_rules else { return Vec::new(); }; - let mut by_domain: std::collections::BTreeMap<&str, Vec<&str>> = + let signed = |e: &crate::record::Entry| { + e.meta + .signed_off_by + .as_deref() + .is_some_and(|by| matches!(Class::of(by, &grants.owner), Class::Owner | Class::Human)) + }; + let mut by_domain: std::collections::BTreeMap<&str, Vec<&crate::record::Entry>> = std::collections::BTreeMap::new(); - for e in entries - .iter() - .filter(|e| e.enforced() && e.meta.signed_off_by.is_none()) - { - by_domain - .entry(e.domain.as_str()) - .or_default() - .push(e.meta.id.as_str()); + for e in entries.iter().filter(|e| { + e.state() == crate::record::Status::Enforced + && !e.meta.is_exception() + && !e.meta.is_core() + && !signed(e) + }) { + by_domain.entry(e.domain.as_str()).or_default().push(e); } let mut over = Vec::new(); for counted in by_domain.values_mut() { - counted.sort_unstable(); - counted.dedup(); - over.extend(counted.iter().skip(ceiling).map(|id| (*id).to_string())); + counted.sort_by(|a, b| (a.created, &a.entry_id).cmp(&(b.created, &b.entry_id))); + over.extend(counted.iter().skip(ceiling).map(|e| e.entry_id.clone())); } over.sort(); over @@ -364,8 +398,11 @@ pub struct Policy { entries: Vec, /// Modules a verified entry carried that would not load, and why. refused: Vec<(String, String)>, - /// The rules the ceiling holds at advisory. + /// The entries of the rules the ceiling holds at advisory. held_back: Vec, + /// The base with the modules of the advisory rules alone, when there are any. What + /// they deny is reported and never joins the decision. + watch: Option, } impl std::fmt::Debug for Policy { @@ -437,9 +474,14 @@ impl Policy { now: time::OffsetDateTime, ) -> Result { let mut texts = self.text.clone(); - let over = over_ceiling(&entries, grants.gates.max_enforced_rules); - // an advisory rule's module is loaded with the enforced ones: it is evaluated - // with every decision, and the base reports what it says without refusing + let over = over_ceiling(&entries, grants); + let advisory = + |e: &crate::record::Entry| e.advisory() || (e.enforced() && over.contains(&e.entry_id)); + // an advisory rule's module is evaluated with every decision, in an engine of + // its own: what it denies is reported, and whatever rule it names it cannot + // refuse + let mut watch = self.engine.clone(); + let mut watched = false; for e in entries.iter().filter(|e| e.enforced() || e.advisory()) { let Some(rego) = &e.rego else { continue }; if !crate::record::module_is_confined(rego) { @@ -447,8 +489,18 @@ impl Policy { .push((e.meta.id.clone(), "not in package amem.rules".into())); continue; } - let mut trial = self.engine.clone(); - match trial.add_policy(format!("{}.rego", e.meta.id), rego.clone()) { + let name = format!("{}.rego", e.entry_id); + let mut trial = if advisory(e) { + watch.clone() + } else { + self.engine.clone() + }; + match trial.add_policy(name, rego.clone()) { + Ok(_) if advisory(e) => { + watch = trial; + watched = true; + texts.push_str(rego); + } Ok(_) => { self.engine = trial; texts.push_str(rego); @@ -456,19 +508,14 @@ impl Policy { Err(err) => self.refused.push((e.meta.id.clone(), err.to_string())), } } + // the rules in force, for what an exception may lift: a rule that is in no + // decision says nothing here, so an unreviewed entry cannot mark a rule as core let policies: Vec<_> = entries .iter() - .filter(|e| !e.meta.is_exception()) + .filter(|e| !e.meta.is_exception() && e.state() == crate::record::Status::Enforced) .map(|e| { - // a rule over the ceiling that nobody signed off is held at advisory - let status = if over.contains(&e.meta.id) { - crate::record::Status::Advisory - } else { - e.state() - }; serde_json::json!({ - "id": e.meta.id, "layer": e.meta.layer, - "core": e.meta.is_core(), "status": status.as_str(), + "id": e.meta.id, "layer": e.meta.layer, "core": e.meta.is_core(), }) }) .collect(); @@ -485,6 +532,11 @@ impl Policy { "grants": grants, "policies": policies, "exceptions": exceptions, }); self.engine = Policy::with_data(self.engine, &data)?; + self.watch = if watched { + Some(Policy::with_data(watch, &data)?) + } else { + None + }; self.version = digest(&texts); self.entries = entries; self.held_back = over; @@ -495,18 +547,63 @@ impl Policy { /// when the ceiling on enforced rules holds it back. #[must_use] pub fn state_of(&self, rule: &crate::record::Entry) -> crate::record::Status { - if self.held_back.contains(&rule.meta.id) { + if self.held_back.contains(&rule.entry_id) { crate::record::Status::Advisory } else { rule.state() } } - /// The rules held at advisory because the grants' ceiling on enforced rules is - /// reached and no person signed them off. + /// The rules held at advisory because the ceiling on enforced rules is reached and + /// no person signed them off, by policy id. #[must_use] - pub fn over_ceiling(&self, grants: &Grants) -> Vec { - over_ceiling(&self.entries, grants.gates.max_enforced_rules) + pub fn over_ceiling(&self) -> Vec { + let mut ids: Vec = self + .entries + .iter() + .filter(|e| self.held_back.contains(&e.entry_id)) + .map(|e| e.meta.id.clone()) + .collect(); + ids.sort(); + ids.dedup(); + ids + } + + /// What the advisory rules deny for a request. An advisory module that fails to + /// evaluate reports that, and changes no decision. + fn advisories(&self, input: &Input) -> Vec { + let Some(watch) = &self.watch else { + return Vec::new(); + }; + let failed = |why: String| { + vec![Denial { + policy: String::new(), + reason: format!("an advisory module did not evaluate: {why}"), + code: Some("advisory-failed".into()), + remedy: None, + }] + }; + let mut engine = watch.clone(); + let value = match serde_json::to_string(input) + .map_err(|e| e.to_string()) + .and_then(|j| regorus::Value::from_json_str(&j).map_err(|e| e.to_string())) + { + Ok(v) => v, + Err(why) => return failed(why), + }; + engine.set_input(value); + let said = engine + .eval_rule("data.amem.rules.deny".to_string()) + .map_err(|e| e.to_string()) + .and_then(|v| v.to_json_str().map_err(|e| e.to_string())) + .and_then(|j| serde_json::from_str::>(&j).map_err(|e| e.to_string())); + match said { + Ok(mut list) => { + list.sort_by(|a, b| (&a.policy, &a.reason).cmp(&(&b.policy, &b.reason))); + list + } + Err(why) => failed(why), + } } fn with_data( @@ -555,6 +652,7 @@ impl Policy { entries: Vec::new(), refused: Vec::new(), held_back: Vec::new(), + watch: None, }) } @@ -610,6 +708,7 @@ impl Policy { let mut d: Decision = serde_json::from_str(&json).map_err(|e| Error::Policy(format!("{e}: {json}")))?; d.policy = self.version.clone(); + d.advisories = self.advisories(input); if !d.allow && !d.denials.is_empty() { let said: Vec = d.denials.iter().map(Denial::said).collect(); d.reason = said.join("; "); @@ -1013,51 +1112,181 @@ mod tests { ); } - /// The ceiling: the rules past it, in the order of their ids, are held at advisory - /// until a person signs one off, and a signature admits that rule alone. + /// A rule written at `day` of November 2026. + fn dated(mut e: crate::record::Entry, day: u8) -> crate::record::Entry { + let at = time::macros::datetime!(2026-11-01 00:00 UTC) + time::Duration::days(day.into()); + e.created = Some(at); + e + } + + /// The ceiling: the enforced rules of a domain past it, oldest first, are held at + /// advisory until a person signs one off, and a signature admits that rule alone. #[test] fn a_rule_over_the_ceiling_is_advisory_until_a_person_signs_it_off() { + use crate::record::Status::{Advisory, Enforced}; use dm_core::TrustTier::Verified; let mut g = grants(); g.gates.max_enforced_rules = Some(1); - // two rules that refuse the same thing; a third that never fires takes a place - let first = rule_entry("pol-L2-001", 2, Verified, "", true); - let second = rule_entry("pol-L2-002", 2, Verified, "", true); - let p = with(&g, vec![second.clone(), first.clone()]); - assert_eq!(p.over_ceiling(&g), ["pol-L2-002"]); + // the newer rule is the one held back, whatever its id: a new rule pushes no + // older one out + let old = dated(rule_entry("pol-L2-900", 2, Verified, "", true), 1); + let new = dated(rule_entry("a-new-rule", 2, Verified, "", true), 2); + let p = with(&g, vec![new.clone(), old.clone()]); + assert_eq!(p.over_ceiling(), ["a-new-rule"]); let d = supersede_decision(&p, None); - assert_eq!(d.cites, ["pol-L2-001"], "the first refuses: {d:?}"); + assert_eq!(d.cites, ["pol-L2-900"], "the older refuses: {d:?}"); assert_eq!(d.advisories.len(), 1); - assert_eq!(d.advisories[0].policy, "pol-L2-002"); - // signed off, the second is enforced, and takes no place under the ceiling - let signed = rule_entry( - "pol-L2-002", - 2, - Verified, - "signed_off_by = \"human:test:lead\"", - true, - ); - let third = rule_entry("pol-L2-003", 2, Verified, "", true); - let p = with(&g, vec![first, signed, third]); + assert_eq!(d.advisories[0].policy, "a-new-rule"); + // signed off by a person, the newer is enforced, and takes no place + let by = |who: &str| { + let extra = format!("signed_off_by = \"{who}\""); + dated(rule_entry("a-new-rule", 2, Verified, &extra, true), 2) + }; + let third = dated(rule_entry("pol-L2-903", 2, Verified, "", true), 3); + let p = with(&g, vec![old.clone(), by("human:test:lead"), third.clone()]); assert_eq!( - p.over_ceiling(&g), - ["pol-L2-003"], + p.over_ceiling(), + ["pol-L2-903"], "the signature raised nothing" ); let mut cites = supersede_decision(&p, None).cites; cites.sort(); - assert_eq!(cites, ["pol-L2-001", "pol-L2-002"]); - assert_eq!(p.state_of(&p.entries()[2]), crate::record::Status::Advisory); - assert_eq!(p.state_of(&p.entries()[0]), crate::record::Status::Enforced); + assert_eq!(cites, ["a-new-rule", "pol-L2-900"]); + assert_eq!(p.state_of(&p.entries()[2]), Advisory); + assert_eq!(p.state_of(&p.entries()[0]), Enforced); + // a sign-off is a person's: an agent's name, or none, admits nothing + for who in ["agent:a", ""] { + let p = with(&g, vec![old.clone(), by(who)]); + assert_eq!(p.over_ceiling(), ["a-new-rule"], "signed by {who:?}"); + } // the ceiling is a domain's: the same rules in two domains are each under it - let mut elsewhere = rule_entry("pol-L2-004", 2, Verified, "", true); + let mut elsewhere = new.clone(); elsewhere.domain = "dom-other".into(); - let here = rule_entry("pol-L2-001", 2, Verified, "", true); - let apart = with(&g, vec![here, elsewhere]); - assert!(apart.over_ceiling(&g).is_empty()); - // with no ceiling, all three refuse + assert!(with(&g, vec![old.clone(), elsewhere]) + .over_ceiling() + .is_empty()); + // a rule with no module is counted as any enforced rule is + let plain = dated(rule_entry("pol-L2-800", 2, Verified, "", false), 0); + assert_eq!( + with(&g, vec![plain, old.clone()]).over_ceiling(), + ["pol-L2-900"] + ); + // with no ceiling, nothing is held back g.gates.max_enforced_rules = None; - assert!(p.over_ceiling(&g).is_empty()); + assert!(with(&g, vec![old, new, third]).over_ceiling().is_empty()); + } + + /// The L0 core is never held back by the ceiling, at any ceiling, and takes no + /// place under it. + #[test] + fn the_ceiling_never_holds_back_the_l0_core() { + use dm_core::TrustTier::Verified; + let mut g = grants(); + let core = dated(rule_entry("pol-L0-001", 0, Verified, "", true), 5); + let older = dated(rule_entry("a-new-rule", 2, Verified, "", true), 1); + for ceiling in [0, 1] { + g.gates.max_enforced_rules = Some(ceiling); + let p = with(&g, vec![core.clone(), older.clone()]); + assert!(!p.over_ceiling().contains(&"pol-L0-001".to_string())); + let d = supersede_decision(&p, None); + assert!( + d.cites.contains(&"pol-L0-001".to_string()), + "{ceiling}: {d:?}" + ); + } + g.gates.max_enforced_rules = Some(1); + assert!(with(&g, vec![core, older]).over_ceiling().is_empty()); + } + + /// A rule with its own module text, at a status. + fn rule_with(id: &str, status: &str, module: &str) -> crate::record::Entry { + let body = format!( + "```policy\nid = \"{id}\"\nlayer = 2\nstatus = \"{status}\"\n```\n\n```rego\npackage amem.rules\nimport rego.v1\n{module}\n```\n" + ); + crate::record::parse( + &format!("mem-p-{id}-{status}"), + dm_core::TrustTier::Verified, + &body, + ) + .unwrap() + } + + /// An advisory module refuses nothing, whatever rule its denial names: another + /// rule's id, an id nobody has, or none at all. + #[test] + fn an_advisory_module_refuses_nothing_whatever_it_names() { + use dm_core::TrustTier::Verified; + let g = grants(); + let enforced = rule_entry("pol-L2-001", 2, Verified, "", false); + for named in [ + "\"policy\": \"pol-L2-001\", ", + "\"policy\": \"nobody\", ", + "", + ] { + let module = format!("deny contains {{{named}\"reason\": \"always\"}} if true"); + let watch = rule_with("pol-L2-050", "advisory", &module); + let p = with(&g, vec![enforced.clone(), watch]); + let d = supersede_decision(&p, None); + assert!(d.allow, "{named}: {d:?}"); + assert!(d.cites.is_empty() && d.denials.is_empty(), "{named}: {d:?}"); + assert_eq!(d.advisories.len(), 1, "{named}: {d:?}"); + } + // a second entry that reuses an enforced rule's id as advisory switches nothing off + let real = rule_entry("pol-L2-031", 2, Verified, "", true); + let shadow = rule_with( + "pol-L2-031", + "advisory", + "deny contains {\"policy\": \"x\"} if false", + ); + let d = supersede_decision(&with(&g, vec![real, shadow]), None); + assert!(!d.allow, "{d:?}"); + assert_eq!(d.cites, ["pol-L2-031"]); + } + + /// A denial is a module's text: a field that is not a string is read as its JSON, + /// and a missing one is empty. The decision is made either way. + #[test] + fn a_malformed_denial_does_not_break_the_decision() { + let g = grants(); + let odd = rule_with( + "pol-L2-060", + "enforced", + "deny contains {\"reason\": \"no id\", \"code\": 7, \"remedy\": [\"a\"]} if true", + ); + let d = supersede_decision(&with(&g, vec![odd.clone()]), None); + assert!(!d.allow, "{d:?}"); + assert_eq!(d.denials[0].code.as_deref(), Some("7")); + assert_eq!(d.denials[0].remedy.as_deref(), Some("[\"a\"]")); + assert_eq!(d.denials[0].policy, ""); + // the same module as advisory: reported, and everything goes ahead + let watch = rule_with( + "pol-L2-060", + "advisory", + odd.rego + .as_deref() + .unwrap() + .split_once("rego.v1\n") + .unwrap() + .1, + ); + let d = supersede_decision(&with(&g, vec![watch]), None); + assert!(d.allow && d.advisories.len() == 1, "{d:?}"); + } + + /// An exception whose block says retired is in force for nothing. + #[test] + fn a_retired_exception_lets_nothing_through() { + use dm_core::TrustTier::Verified; + let g = grants(); + let rule = rule_entry("pol-L2-031", 2, Verified, "", true); + let exc = rule_entry( + "exc-2026-020", + 2, + Verified, + "breaches = \"pol-L2-031\"\nscope_task = \"task-t-1\"\nexpires = \"2099-01-01T00:00:00Z\"\nstatus = \"retired\"", + false, + ); + assert!(!supersede_decision(&with(&g, vec![rule, exc]), Some("task-t-1")).allow); } /// The core of L0 takes no exception; an L0 rule marked as outside the core takes one. diff --git a/crates/policy/src/record.rs b/crates/policy/src/record.rs index 1480d09a..c734cc66 100644 --- a/crates/policy/src/record.rs +++ b/crates/policy/src/record.rs @@ -205,6 +205,9 @@ pub struct Entry { /// The domain its entry is in; empty for one that came from no record. #[serde(default)] pub domain: String, + /// When its entry was written, where the record says. + #[serde(skip)] + pub created: Option, } impl Entry { @@ -237,10 +240,12 @@ impl Entry { self.state() == Status::Advisory && self.rego.is_some() && !self.meta.is_exception() } - /// Whether, as an exception, it is in force at `now`: verified, not expired. + /// Whether, as an exception, it is in force at `now`: verified, not expired, and + /// not retired. #[must_use] pub fn in_force(&self, now: time::OffsetDateTime) -> bool { self.meta.is_exception() + && self.meta.status != Some(Status::Retired) && self.tier == dm_core::TrustTier::Verified && self.meta.expires.as_deref().is_some_and(|e| { time::OffsetDateTime::parse(e, &time::format_description::well_known::Rfc3339) @@ -274,6 +279,7 @@ pub fn parse(entry_id: &str, tier: dm_core::TrustTier, body: &str) -> Result Vec { ), }); } - if let Some(said) = m.status { - if said != state { - out.push(Finding { - policy: m.id.clone(), - kind: "held-back".into(), - why: format!( - "{} says {} and its entry is {}: it is {} until the entry is reviewed that far", - m.id, - said.as_str(), - e.tier, - state.as_str() - ), - }); - } - } out } diff --git a/crates/server/src/lib.rs b/crates/server/src/lib.rs index 6ce51c19..46e8336e 100644 --- a/crates/server/src/lib.rs +++ b/crates/server/src/lib.rs @@ -236,6 +236,7 @@ fn policy_entries(ix: &Index, root: &Path, grants: &Grants) -> Vec Option { + async fn status_of(root: &Path, id: &str) -> Option { let m = Memory::open(root).unwrap(); m.check_policy("supersede a decision", None) + .await .unwrap() .into_iter() .find(|r| r.policy_id == id) @@ -6469,13 +6471,16 @@ mod tests { .promote(&rule, TrustTier::Curated, "checked") .await .unwrap(); - assert_eq!(status_of(dir.path(), "pol-L2-050"), None); + assert_eq!(status_of(dir.path(), "pol-L2-050").await, None); let d2 = agent_supersedes(dir.path(), &d1).await.unwrap().id; let log = || std::fs::read_to_string(dir.path().join("state/decisions.jsonl")).unwrap(); assert!(!log().contains("pol-L2-050"), "in no decision yet"); // a person verifies it: advisory, which reports and refuses nothing verify(&owner, &curator, &rule).await; - assert_eq!(status_of(dir.path(), "pol-L2-050"), Some(Rule::Advisory)); + assert_eq!( + status_of(dir.path(), "pol-L2-050").await, + Some(Rule::Advisory) + ); let d3 = agent_supersedes(dir.path(), &d2).await.unwrap().id; let all = log(); let reported = all.lines().last().unwrap_or_default(); @@ -6496,9 +6501,16 @@ mod tests { .await .unwrap() .id; - assert_eq!(status_of(dir.path(), "pol-L2-050"), None, "under review"); + assert_eq!( + status_of(dir.path(), "pol-L2-050").await, + None, + "under review" + ); verify(&owner, &curator, &enforced).await; - assert_eq!(status_of(dir.path(), "pol-L2-050"), Some(Rule::Enforced)); + assert_eq!( + status_of(dir.path(), "pol-L2-050").await, + Some(Rule::Enforced) + ); let refused = agent_supersedes(dir.path(), &d3) .await .unwrap_err() @@ -6543,7 +6555,6 @@ mod tests { scope: "this one replacement".into(), expires: expires.into(), justification: "The task is to replace the decision.".into(), - domain: None, }; let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); let core = agent @@ -6610,10 +6621,12 @@ mod tests { assert!(matches!(unknown, Err(Error::NotFound(_)))); } - /// A reset keeps each rule that a person wrote or promoted and retires the rest, - /// and only the owner resets. + /// A reset keeps each rule that a person wrote, curated or approved and retires the + /// rest. Only the owner resets. A proposal that a person made and nobody approved + /// puts no person in the lineage, and an exception is not touched. #[tokio::test] async fn a_reset_keeps_the_rules_with_a_person_in_their_lineage() { + use dm_policy::record::Status as Rule; let dir = tempfile::tempdir().unwrap(); let owner = mem(dir.path()); let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); @@ -6621,7 +6634,8 @@ mod tests { .unwrap() .with_author("agent:curator"); // written by a person; written by an agent and verified by a person; written by - // an agent and curated by an agent; written by an agent and never reviewed + // an agent and curated by an agent; written by an agent and never reviewed; + // written by an agent and proposed by a person whose proposal was declined policy_entry(&owner, &rule_text("pol-L3-001", 3, "")).await; let verified = policy_entry(&agent, &rule_text("pol-L3-002", 3, "")).await; verify(&owner, &curator, &verified).await; @@ -6631,6 +6645,25 @@ mod tests { .await .unwrap(); policy_entry(&agent, &rule_text("pol-L3-004", 3, "")).await; + let declined = policy_entry(&agent, &rule_text("pol-L3-005", 3, "")).await; + let other = Memory::open(dir.path()) + .unwrap() + .with_author("human:test:other"); + let Promotion::Proposed { task } = other + .promote(&declined, TrustTier::Verified, "looks right") + .await + .unwrap() + else { + panic!("proposed") + }; + owner.invalidate(&task.id, "declined", None).await.unwrap(); + let exception = "# Exception\n\n```policy\nid = \"exc-2026-030\"\nlayer = 3\nbreaches = \"pol-L3-002\"\nscope_task = \"task-me-x\"\nexpires = \"2099-01-01T00:00:00Z\"\n```\n"; + let exception = policy_entry(&agent, exception).await; + // a value that an agent wrote and nobody reviewed is not in the digest as core + let value = "# A value\n\n```policy\nid = \"pol-L0-102\"\nlayer = 0\nrule = \"An agent's value.\"\nintent = \"i\"\n```\n"; + policy_entry(&agent, value).await; + let digest = Memory::open(dir.path()).unwrap().t0(None).await.unwrap(); + assert!(!digest.contains("pol-L0-102"), "{digest}"); let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); assert!(matches!( agent.reset_rules(None).await, @@ -6640,17 +6673,228 @@ mod tests { let reset = owner.reset_rules(None).await.unwrap(); let kept: Vec<_> = reset.kept.iter().map(|k| k.policy_id.as_str()).collect(); assert_eq!(kept, ["pol-L3-001", "pol-L3-002"]); - assert_eq!(reset.retired, ["pol-L3-003", "pol-L3-004"]); + let retired: Vec<_> = reset.retired.iter().map(|r| r.policy_id.as_str()).collect(); + assert_eq!( + retired, + ["pol-L0-102", "pol-L3-003", "pol-L3-004", "pol-L3-005"] + ); + let digest = Memory::open(dir.path()).unwrap().t0(None).await.unwrap(); + assert!(!digest.contains("pol-L0-102"), "{digest}"); + assert_eq!(reset.retired[1].entry, curated); assert!(reset.kept.iter().all(|k| k.person == "human:test:me")); - assert_eq!(status_of(dir.path(), "pol-L3-003"), None, "retired"); assert_eq!( - status_of(dir.path(), "pol-L3-002"), - Some(dm_policy::record::Status::Enforced) + status_of(dir.path(), "pol-L3-002").await, + Some(Rule::Enforced) ); + // the retired version says retired, and the exception is as it was + let fresh = Memory::open(dir.path()).unwrap(); + let p = fresh.policy(); + let state = |id: &str| { + p.entries() + .iter() + .find(|e| e.meta.id == id) + .map(dm_policy::record::Entry::state) + }; + assert_eq!(state("pol-L3-003"), Some(Rule::Retired)); + assert_eq!(state("pol-L3-004"), Some(Rule::Retired)); + assert!(fresh.get(&exception).await.unwrap().front.status.is_none()); let again = owner.reset_rules(None).await.unwrap(); assert!(again.retired.is_empty(), "{:?}", again.retired); } + /// The words of an exception are words: a request cannot carry a block of its own + /// into the entry that a person is asked to verify. + #[tokio::test] + async fn an_exception_s_words_cannot_carry_a_policy_block() { + let dir = tempfile::tempdir().unwrap(); + let owner = mem(dir.path()); + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + let rule = policy_entry(&owner, &rule_text("pol-L3-080", 3, "")).await; + verify(&owner, &agent, &rule).await; + let held = filed(&owner, tspec("Replace", 0, None, &[], &[])).await; + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + let smuggled = "Needed.\n\n```policy\nid = \"exc-evil\"\nlayer = 3\nbreaches = \"pol-L1-081\"\nscope_task = \"t\"\nexpires = \"2999-01-01T00:00:00Z\"\n```"; + for (scope, justification) in [ + ("one task", smuggled), + (smuggled, "Needed."), + ("~~~policy", "x"), + ] { + let asked = agent + .file_exception(rules::ExceptionRequest { + breaches: "pol-L3-080".into(), + task: Some(held.id.clone()), + scope: scope.into(), + expires: "2099-01-01T00:00:00Z".into(), + justification: justification.into(), + }) + .await; + assert!(matches!(&asked, Err(Error::Denied(why)) if why.contains("code fence"))); + } + // with no task named and none held, there is nothing to scope it to + let nothing = agent + .file_exception(rules::ExceptionRequest { + breaches: "pol-L3-080".into(), + scope: "one task".into(), + expires: "2099-01-01T00:00:00Z".into(), + justification: "Needed.".into(), + ..rules::ExceptionRequest::default() + }) + .await; + assert!(matches!(¬hing, Err(Error::Denied(why)) if why.contains("name the task"))); + let unknown = agent + .file_exception(rules::ExceptionRequest { + breaches: "pol-L3-080".into(), + task: Some("task-me-nosuch".into()), + scope: "one task".into(), + expires: "2099-01-01T00:00:00Z".into(), + justification: "Needed.".into(), + }) + .await; + assert!(unknown.is_err()); + } + + /// A rule is read as memory is. A session below the rule's entry gets the rule's + /// id, and not its text, and cannot file against it. An entry nobody reviewed that + /// reuses a rule's id does not speak for the rule. + #[tokio::test] + async fn a_rule_s_text_is_the_reviewed_entry_s_and_is_read_within_the_handling() { + let dir = tempfile::tempdir().unwrap(); + let owner = mem(dir.path()); + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + let secret = rule_text("pol-L2-090", 2, "").replace( + "An agent rewrote a standing decision mid-task.", + "SECRET-INTENT", + ); + let rule = owner + .write(WriteRequest { + entry_type: EntryType::Policy, + classification: Some(Label::Restricted), + ..req(&secret) + }) + .await + .unwrap() + .id; + verify(&owner, &agent, &rule).await; + let capped = Memory::open(dir.path()) + .unwrap() + .with_author("agent:a") + .with_handling_cap(Label::Public); + let rules = capped.check_policy("anything", None).await.unwrap(); + assert_eq!(rules.len(), 1); + assert!(rules[0].withheld && rules[0].intent.is_empty() && rules[0].rule.is_empty()); + assert_eq!(rules[0].policy_id, "pol-L2-090"); + assert_eq!(capped.label(), Label::Public); + let amend = capped.propose_amendment("pol-L2-090", "It is wrong.").await; + assert!(matches!(amend, Err(Error::NotFound(_)))); + // an agent's raw entry with the same id, another owner and another text + let fake = secret + .replace("human:test:lead", "human:evil:x") + .replace("SECRET-INTENT", "FAKE"); + policy_entry(&agent, &fake).await; + let reader = Memory::open(dir.path()).unwrap().with_author("agent:b"); + let filed = reader + .propose_amendment("pol-L2-090", "It is wrong.") + .await + .unwrap(); + assert!(filed.task.spec.body.contains("human:test:lead")); + assert!(!filed.task.spec.body.contains("human:evil:x")); + assert_eq!(filed.task.spec.classification, Label::Restricted); + // one open amendment for each rule and filer: other evidence is the same task + let again = reader + .propose_amendment("pol-L2-090", "It is wrong in another case.") + .await + .unwrap(); + assert!(again.deduplicated); + assert_eq!(again.task.id, filed.task.id); + let rules = reader.check_policy("anything", None).await.unwrap(); + assert_eq!(rules[0].intent, "SECRET-INTENT"); + assert_eq!(rules[0].owner.as_deref(), Some("human:test:lead")); + } + + /// The lint: a rule over the ceiling, a rule that refused nothing, a rule with no + /// owner and a rule after its review date are findings, and the rates count what + /// an advisory rule would have refused. + #[tokio::test] + async fn the_lint_finds_what_the_lifecycle_asks_of_a_rule() { + let dir = tempfile::tempdir().unwrap(); + let owner = mem(dir.path()); + let agent = Memory::open(dir.path()).unwrap().with_author("agent:a"); + let d1 = owner + .write(WriteRequest { + entry_type: EntryType::Decision, + ..req("Use SQLite on a laptop.\n") + }) + .await + .unwrap() + .id; + let grants = dir.path().join(GRANTS_FILE); + let text = std::fs::read_to_string(&grants).unwrap(); + let ceiling = if text.contains("[gates]") { + text.replace("[gates]", "[gates]\nmax_enforced_rules = 1") + } else { + format!("{text}\n[gates]\nmax_enforced_rules = 1\n") + }; + std::fs::write(&grants, ceiling).unwrap(); + let owner = Memory::open(dir.path()).unwrap(); + let first = policy_entry(&owner, &rule_text("pol-L2-001", 2, "")).await; + verify(&owner, &agent, &first).await; + let late = rule_text("pol-L2-002", 2, "review_by = \"2020-01-01\"\n") + .replace("owner = \"human:test:lead\"\n", ""); + let second = policy_entry(&owner, &late).await; + verify(&owner, &agent, &second).await; + // the first refuses, and the second, over the ceiling, advises + assert!(agent_supersedes(dir.path(), &d1).await.is_err()); + let m = Memory::open(dir.path()).unwrap(); + let kinds = |id: &str| { + let mut kinds: Vec = m + .policy_findings() + .into_iter() + .filter(|f| f.policy == id) + .map(|f| f.kind) + .collect(); + kinds.sort(); + kinds + }; + assert!(kinds("pol-L2-001").is_empty(), "{:?}", kinds("pol-L2-001")); + assert_eq!( + kinds("pol-L2-002"), + ["no-owner", "over-ceiling", "review-due"] + ); + let rates = m.policy_rates(); + let rate = |id: &str| { + rates + .iter() + .find(|r| r.id == id) + .cloned() + .unwrap_or_default() + }; + assert_eq!( + (rate("pol-L2-001").refused, rate("pol-L2-001").advised), + (1, 0) + ); + assert_eq!( + (rate("pol-L2-002").refused, rate("pol-L2-002").advised), + (0, 1) + ); + // without the ceiling the second is enforced, and it has refused nothing yet + std::fs::write(&grants, text).unwrap(); + let m = Memory::open(dir.path()).unwrap(); + let unused: Vec = m + .policy_findings() + .into_iter() + .filter(|f| f.kind == "unused") + .map(|f| f.policy) + .collect(); + assert_eq!(unused, ["pol-L2-002"]); + let filed = m.file_policy_findings(&m.policy_findings()).await.unwrap(); + assert_eq!(filed.len(), 3, "no-owner, review-due and unused"); + assert!(m + .file_policy_findings(&m.policy_findings()) + .await + .unwrap() + .is_empty()); + } + async fn an_exception_lets_one_task_through( root: &Path, owner: &Memory, diff --git a/crates/server/src/mcp.rs b/crates/server/src/mcp.rs index 0b384a3e..fa696f3f 100644 --- a/crates/server/src/mcp.rs +++ b/crates/server/src/mcp.rs @@ -429,9 +429,6 @@ pub struct FileExceptionParams { pub expiry: String, /// Why it is needed. pub justification: String, - /// The domain to write it in; the only granted domain when omitted. - #[serde(default)] - pub domain: Option, } /// Parameters of `propose_amendment`. @@ -1017,7 +1014,7 @@ impl McpServer { } #[tool( - description = "The rules in force for what you are about to do: each with its id, layer, rule, intent and whether it is enforced or advisory. Call it before an action that cannot be taken back." + description = "The rules in force for what you are about to do: each with its id, layer, rule, intent and whether it is enforced or advisory. Call it before an action that cannot be taken back. A rule above what you may read is named, and its text is withheld." )] async fn check_policy( &self, @@ -1027,12 +1024,13 @@ impl McpServer { let rules = self .memory .check_policy(&p.action, p.domain.as_deref()) + .await .map_err(mcp_err)?; ok(json!({ "rules": rules })) } #[tool( - description = "Ask for an exception to a rule that refused you: for one task, until an expiry, with a justification. It lets nothing through until a person approves it. A rule of the L0 core takes no exception." + description = "Ask for an exception to a rule that refused you: for one task, until an expiry, with a justification. The scope and the justification are plain words. It lets nothing through until a person approves it. A rule of the L0 core takes no exception." )] async fn file_exception( &self, @@ -1047,7 +1045,6 @@ impl McpServer { scope: p.scope, expires: p.expiry, justification: p.justification, - domain: p.domain, }) .await .map_err(mcp_err)?; @@ -1551,7 +1548,6 @@ mod tests { scope: "one test tag".into(), expiry: "2099-01-01T00:00:00Z".into(), justification: "The tag is on a scratch repository.".into(), - domain: None, })) .await .unwrap(), diff --git a/crates/server/src/rules.rs b/crates/server/src/rules.rs index 0fdef040..2dc31e99 100644 --- a/crates/server/src/rules.rs +++ b/crates/server/src/rules.rs @@ -1,6 +1,6 @@ //! The rule lifecycle as a session meets it: the rules in force where it is about to -//! act, an exception asked for, an amendment proposed to a rule's owner, and the reset -//! by which a domain's owner retires what no person stands behind. +//! act, an exception asked for, an amendment proposed to a rule's owner, the reset by +//! which a domain's owner retires what no person stands behind, and the lint. //! //! A rule is a policy entry ([`dm_policy::record`]). Where it is in its life is its //! block's `status` held back by its entry's tier, so every step up is a reviewed @@ -11,8 +11,9 @@ use crate::{Error, Memory, WriteRequest}; use dm_core::{EntryType, TrustTier}; use dm_ledger::{Task, TaskSpec}; use dm_policy::decide::Class; -use dm_policy::record::{self, Meta, Status}; +use dm_policy::record::{self, Finding, Meta, Status}; use serde::{Deserialize, Serialize}; +use std::collections::{BTreeMap, HashMap}; use time::OffsetDateTime; /// The origin of a task that proposes an amendment to a rule, before the rule's id. @@ -25,9 +26,9 @@ pub struct RuleInForce { pub policy_id: String, /// 0 values, 1 strategy, 2 practice, 3 domain convention. pub layer: u8, - /// The rule, one sentence. + /// The rule, one sentence. Empty when withheld. pub rule: String, - /// Why: what an edge case is resolved from. + /// Why: what an edge case is resolved from. Empty when withheld. pub intent: String, /// `enforced`, which refuses, or `advisory`, which reports. pub status: Status, @@ -36,6 +37,10 @@ pub struct RuleInForce { /// Who answers for it. #[serde(skip_serializing_if = "Option::is_none")] pub owner: Option, + /// The rule's entry is above what this session may read: the rule binds, and its + /// text is not given. + #[serde(skip_serializing_if = "std::ops::Not::not")] + pub withheld: bool, } /// What a session asks an exception for. @@ -52,9 +57,6 @@ pub struct ExceptionRequest { pub expires: String, /// Why it is needed. pub justification: String, - /// The domain to write it in; the only granted domain when omitted. - #[serde(default)] - pub domain: Option, } /// An exception as filed: written, in force for nothing until a person approves it. @@ -64,7 +66,7 @@ pub struct FiledException { pub id: String, /// Its policy id, which a decision that uses it cites. pub policy_id: String, - /// Who approves: the higher the layer breached, the more senior. + /// Who should approve: the higher the layer breached, the more senior. pub approver: String, /// The approval task. pub task: Box, @@ -77,8 +79,8 @@ pub struct Reset { pub domain: String, /// The rules kept, each with the person in its lineage. pub kept: Vec, - /// The rules retired, by policy id. - pub retired: Vec, + /// The rules retired. + pub retired: Vec, } /// A rule a reset kept. @@ -86,67 +88,134 @@ pub struct Reset { pub struct Kept { /// The rule. pub policy_id: String, - /// The person who wrote it or promoted it. + /// The entry that holds it. + pub entry: String, + /// The person who wrote it, curated it or approved it. pub person: String, } +/// A rule a reset retired. +#[derive(Clone, Debug, PartialEq, Eq, Serialize)] +pub struct Retired { + /// The rule. + pub policy_id: String, + /// The entry that held it, which the retired version supersedes. + pub entry: String, +} + +/// How often a rule was cited, from the decision log. +#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize)] +pub struct Rate { + /// The policy id. + pub id: String, + /// Decisions it refused. + pub refused: u64, + /// Decisions an exception let through, for an exception's id. + pub let_through: u64, + /// Decisions it would have refused, as an advisory rule. + pub advised: u64, +} + fn is_person(author: &str, owner: &str) -> bool { matches!(Class::of(author, owner), Class::Owner | Class::Human) } +fn live(status: Status) -> bool { + matches!(status, Status::Enforced | Status::Advisory) +} + impl Memory { /// The rules in force for an action in a domain: every enforced and advisory rule /// of layers 0 to 2, and the domain's own conventions. A session asks before an /// action it cannot take back. The rules are not chosen by the action: the action - /// is what the read log keeps, so an audit finds that the session asked. + /// is what the read log keeps, so an audit finds that the session asked. A rule is + /// read as memory is: one whose entry this session may not read is named, with its + /// layer and status, and its text is withheld. /// /// # Errors /// - /// A domain outside the grants. - pub fn check_policy( + /// A domain outside the grants, or an index error. + pub async fn check_policy( &self, action: &str, domain: Option<&str>, ) -> Result, Error> { let domain = self.resolve_domain(domain)?; + let readable: Vec = self + .list(None, Some(EntryType::Policy), 1000) + .await? + .into_iter() + .map(|h| h.id) + .collect(); let p = self.policy(); let mut rules: Vec = p .entries() .iter() .filter(|e| !e.meta.is_exception()) .filter(|e| e.meta.layer < 3 || e.domain == domain) - .filter_map(|e| { - let status = p.state_of(e); - matches!(status, Status::Enforced | Status::Advisory).then(|| RuleInForce { + .filter(|e| live(p.state_of(e))) + .map(|e| { + let withheld = !readable.contains(&e.entry_id); + let text = |s: &str| { + if withheld { + String::new() + } else { + s.to_string() + } + }; + RuleInForce { policy_id: e.meta.id.clone(), layer: e.meta.layer, - rule: e.meta.rule.clone(), - intent: e.meta.intent.clone(), - status, + rule: text(&e.meta.rule), + intent: text(&e.meta.intent), + status: p.state_of(e), checked: e.rego.is_some(), - owner: e.meta.owner.clone(), - }) + owner: e.meta.owner.clone().filter(|_| !withheld), + withheld, + } }) .collect(); - rules.sort_by(|a, b| (a.layer, &a.policy_id).cmp(&(b.layer, &b.policy_id))); + // an enforced entry of an id before an advisory one of the same id + rules.sort_by(|a, b| { + (a.layer, &a.policy_id, a.status != Status::Enforced).cmp(&( + b.layer, + &b.policy_id, + b.status != Status::Enforced, + )) + }); rules.dedup_by(|a, b| a.policy_id == b.policy_id); self.reads .text("check_policy", &format!("{action} in {domain}")); Ok(rules) } - /// The rule of a policy id, among the record's rules. - fn rule(&self, policy_id: &str) -> Result { - self.policy() + /// The rule of a policy id: the entry in force when one is, so an entry nobody + /// reviewed that reuses the id does not speak for the rule. It is read as this + /// session, so a rule the session may not read is not found. + async fn rule(&self, policy_id: &str) -> Result<(record::Entry, dm_core::Entry), Error> { + let p = self.policy(); + let rank = |e: &record::Entry| match (p.state_of(e), e.tier) { + (Status::Enforced, _) => 0, + (Status::Advisory, _) => 1, + (_, TrustTier::Verified) => 2, + _ => 3, + }; + let found = p .entries() .iter() - .find(|e| e.meta.id == policy_id && !e.meta.is_exception()) + .filter(|e| e.meta.id == policy_id && !e.meta.is_exception()) + .min_by_key(|e| (rank(e), e.entry_id.clone())) .cloned() - .ok_or_else(|| Error::NotFound(format!("policy {policy_id}"))) + .ok_or_else(|| Error::NotFound(format!("policy {policy_id}")))?; + let entry = self + .get(&found.entry_id) + .await + .map_err(|_| Error::NotFound(format!("policy {policy_id}")))?; + Ok((found, entry)) } - /// Who approves an exception to a rule: the rule's owner for a domain convention or - /// a practice, and the owner of the record for strategy and values. + /// Who should approve an exception to a rule: the rule's owner for a domain + /// convention or a practice, and the owner of the record for strategy and values. fn approver_of(&self, rule: &record::Entry) -> String { match (&rule.meta.owner, rule.meta.layer) { (Some(owner), 2..) => owner.clone(), @@ -154,23 +223,18 @@ impl Memory { } } - /// Ask for an exception to a rule: for one task, until a time, with a reason. The - /// exception is written as a policy entry and proposed for approval; it lets - /// nothing through until a person approves it, and after its expiry it lets - /// nothing through again. - /// - /// # Errors - /// - /// [`Error::NotFound`] for a rule or a task that is not there; [`Error::Denied`] - /// for a rule of the L0 core, an expiry that is not a time to come, or no task to - /// scope it to; and as [`Memory::write`] and [`Memory::promote`]. - pub async fn file_exception(&self, ask: ExceptionRequest) -> Result { - let rule = self.rule(&ask.breaches)?; - if rule.meta.is_core() { - return Err(Error::Denied(format!( - "{} is a rule of the L0 core, which takes no exception", - rule.meta.id - ))); + /// What a request for an exception must be before anything is written: words with + /// no code fence, an expiry that is a time to come, and a task to hold for. Gives + /// the task and the time of the check. + fn exception_asked(&self, ask: &ExceptionRequest) -> Result<(String, OffsetDateTime), Error> { + if [&ask.scope, &ask.justification] + .iter() + .any(|words| words.contains("```") || words.contains("~~~")) + { + return Err(Error::Denied( + "the scope and the justification of an exception are words, with no code fence" + .into(), + )); } let now = OffsetDateTime::now_utc(); let rfc3339 = &time::format_description::well_known::Rfc3339; @@ -185,11 +249,37 @@ impl Memory { "an exception holds for one task: name the task".into(), )); }; + Ok((task, now)) + } + + /// Ask for an exception to a rule: for one task, until a time, with a reason. The + /// exception is written as a policy entry in the rule's domain and proposed for + /// approval; it lets nothing through until a person who did not file it approves + /// it, and after its expiry it lets nothing through again. + /// + /// The words of the request go into an entry a person is asked to verify, so they + /// are words only: a request that carries a code fence is refused, and the entry + /// is written only if it reads back as the exception that was asked for. + /// + /// # Errors + /// + /// [`Error::NotFound`] for a rule or a task that is not there for this session; + /// [`Error::Denied`] for a rule of the L0 core, an expiry that is not a time to + /// come, no task to scope it to, or a code fence in the words; and as + /// [`Memory::write`] and [`Memory::promote`]. + pub async fn file_exception(&self, ask: ExceptionRequest) -> Result { + let (rule, rule_entry) = self.rule(&ask.breaches).await?; + if rule.meta.is_core() { + return Err(Error::Denied(format!( + "{} is a rule of the L0 core, which takes no exception", + rule.meta.id + ))); + } + let (task, now) = self.exception_asked(&ask)?; self.get_task(&task).await?; let approver = self.approver_of(&rule); let key = dm_core::digest(&format!("{}\n{task}\n{}", rule.meta.id, ask.expires)); - let hex = key.rsplit(':').next().unwrap_or_default(); - let policy_id = format!("exc-{}-{}", now.year(), hex.get(..8).unwrap_or(hex)); + let policy_id = format!("exc-{}-{}", now.year(), key.get(..8).unwrap_or(&key)); let meta = Meta { id: policy_id.clone(), layer: rule.meta.layer, @@ -204,9 +294,15 @@ impl Memory { }; let title = format!("Exception to {} for {task}", rule.meta.id); let body = record::render(&title, &ask.justification, &meta).map_err(Error::Denied)?; + let reads_back = record::parse("", TrustTier::Raw, &body); + if !reads_back.is_ok_and(|e| e.meta == meta && e.rego.is_none()) { + return Err(Error::Denied( + "the exception does not read back as it was asked for, and is not written".into(), + )); + } let written = self .write(WriteRequest { - domain: ask.domain.clone(), + domain: Some(rule_entry.front.domain.clone()), entry_type: EntryType::Policy, title: Some(title), body, @@ -233,35 +329,46 @@ impl Memory { } /// Propose an amendment to a rule: an ordinary task for a person, which names the - /// rule's owner and carries the evidence. The same evidence against the same rule - /// is one task. + /// rule's owner and carries the evidence. It is classified as the rule's entry is. + /// A filer has one open amendment to a rule at a time: asked again, it gets that + /// task back. /// /// # Errors /// - /// [`Error::NotFound`] for a rule that is not there; [`Error::Denied`] for no - /// evidence; a ledger error, or a filing the policy refuses. + /// [`Error::NotFound`] for a rule that is not there for this session; + /// [`Error::Denied`] for no evidence; a ledger error, or a filing the policy + /// refuses. pub async fn propose_amendment( &self, policy_id: &str, evidence: &str, ) -> Result { - let rule = self.rule(policy_id)?; + let (rule, entry) = self.rule(policy_id).await?; if evidence.trim().is_empty() { return Err(Error::Denied( "an amendment carries its evidence: what happened, and the entries that show it" .into(), )); } + let domain = entry.front.domain.clone(); + let origin = format!("{AMENDMENT_ORIGIN}{}", rule.meta.id); + for status in [dm_ledger::Status::Proposed, dm_ledger::Status::Open] { + let open = self.tasks(Some(&domain), status).await?; + if let Some(task) = open.into_iter().find(|t| { + t.spec.origin.as_deref() == Some(origin.as_str()) + && t.spec.created_by == self.author + }) { + return Ok(dm_ledger::Created { + task, + deduplicated: true, + }); + } + } let owner = rule .meta .owner .clone() .unwrap_or_else(|| self.grants.owner.clone()); - let domain = if rule.domain.is_empty() { - self.resolve_domain(None)? - } else { - rule.domain.clone() - }; let body = format!( "For {owner}, who owns {id}.\n\nThe rule: {rule}\n\nIts intent: {intent}\n\n\ The evidence:\n\n{evidence}\n\nAmend the rule by superseding its entry `{entry}`, \ @@ -277,47 +384,53 @@ impl Memory { title: format!("Amend {}", rule.meta.id), body, requirements: vec!["human".into()], - origin: Some(format!("{AMENDMENT_ORIGIN}{}", rule.meta.id)), - upstream_event: Some(dm_core::digest(evidence.trim())), + origin: Some(origin), + upstream_event: Some(dm_core::digest(&format!( + "{}\n{}", + self.author, + evidence.trim() + ))), + classification: entry.front.classification, priority: 3, ..TaskSpec::default() }) .await } - /// The person in a rule's lineage: who wrote its entry, who promoted it, or who - /// approved its promotion. + /// The person in a rule's lineage: who wrote its entry, who curated it, or who + /// approved its promotion to verified. A proposal that nobody approved puts nobody + /// in the lineage. async fn person_behind(&self, rule: &record::Entry) -> Result, Error> { let owner = &self.grants.owner; let entry = self.get(&rule.entry_id).await?; if is_person(&entry.front.author, owner) { return Ok(Some(entry.front.author)); } - // a promotion to curated names who made it; a promotion to verified names who - // proposed it, and the approval task it closed names who approved let l = self.ledger.lock().await; let promoted = l.promotions(&rule.domain, &rule.entry_id).await?; + let curators = promoted + .into_iter() + .filter(|p| p.spec.tier == TrustTier::Curated && entry.front.trust_tier >= p.spec.tier) + .map(|p| p.spec.by); let approvals = format!("{}{}", crate::PROPOSAL_ORIGIN, rule.entry_id); let approved = l.list(&rule.domain, dm_ledger::Status::Done).await?; let approvers = approved .into_iter() .filter(|t| t.spec.origin.as_deref() == Some(approvals.as_str())) .filter_map(|t| t.holder); - Ok(promoted - .into_iter() - .map(|p| p.spec.by) - .chain(approvers) - .find(|by| is_person(by, owner))) + Ok(curators.chain(approvers).find(|by| is_person(by, owner))) } /// Reset a domain's rules: the owner retires, in one step, every rule that no - /// person wrote or promoted, and keeps the rest. A retired rule is superseded by a - /// version whose block says so, as every change to the record is a version. + /// person wrote, curated or approved, and keeps the rest. A retired rule is + /// superseded by a version whose block says so, as every change to the record is a + /// version. Exceptions and the rules of other domains are not touched. /// /// # Errors /// /// [`Error::Denied`] for anyone but the owner of the record, or a domain outside - /// the grants; and as [`Memory::write`]. + /// the grants; and as [`Memory::write`]. After an error, what was retired before + /// it stays retired, and a second reset does the rest. pub async fn reset_rules(&self, domain: Option<&str>) -> Result { if self.author != self.grants.owner { return Err(Error::Denied(format!( @@ -337,6 +450,7 @@ impl Memory { if let Some(person) = self.person_behind(rule).await? { out.kept.push(Kept { policy_id: rule.meta.id.clone(), + entry: rule.entry_id.clone(), person, }); continue; @@ -352,11 +466,135 @@ impl Memory { ..WriteRequest::default() }) .await?; - out.retired.push(rule.meta.id.clone()); + out.retired.push(Retired { + policy_id: rule.meta.id.clone(), + entry: rule.entry_id.clone(), + }); } - out.kept.sort_by(|a, b| a.policy_id.cmp(&b.policy_id)); - out.retired.sort(); + out.kept + .sort_by(|a, b| (&a.policy_id, &a.entry).cmp(&(&b.policy_id, &b.entry))); + out.retired + .sort_by(|a, b| (&a.policy_id, &a.entry).cmp(&(&b.policy_id, &b.entry))); self.reload_policy().await?; Ok(out) } + + /// Citations per policy, from the decision log: how often each rule refused, how + /// often an exception let something through, and how often an advisory rule would + /// have refused. + #[must_use] + pub fn policy_rates(&self) -> Vec { + let text = + std::fs::read_to_string(self.root.join("state/decisions.jsonl")).unwrap_or_default(); + let mut rates: BTreeMap = BTreeMap::new(); + let names = |list: &serde_json::Value, key: Option<&str>| -> Vec { + list.as_array() + .into_iter() + .flatten() + .filter_map(|x| key.map_or(x.as_str(), |k| x[k].as_str())) + .map(str::to_string) + .collect() + }; + for line in text.lines() { + let Ok(v) = serde_json::from_str::(line) else { + continue; + }; + let allow = v["allow"].as_bool().unwrap_or(false); + for id in names(&v["cites"], None) { + let rate = rates.entry(id).or_default(); + if allow { + rate.let_through += 1; + } else { + rate.refused += 1; + } + } + for id in names(&v["advisories"], Some("policy")) { + rates.entry(id).or_default().advised += 1; + } + } + for (id, rate) in &mut rates { + rate.id.clone_from(id); + } + rates.into_values().collect() + } + + /// The lint over the record's rules: the paper's conflict rules, a module that + /// would not load, a rule in force with no owner or past its review date, a rule + /// the ceiling holds at advisory, and an enforced rule that no decision in the log + /// cites, which goes to its owner to retire or to say why it stays. + #[must_use] + pub fn policy_findings(&self) -> Vec { + let p = self.policy(); + let mut findings = record::lint(p.entries(), OffsetDateTime::now_utc()); + findings.extend(p.refused().iter().map(|(id, why)| Finding { + policy: id.clone(), + kind: "module-refused".into(), + why: format!("{id}'s module would not load: {why}"), + })); + findings.extend(p.over_ceiling().into_iter().map(|id| Finding { + why: format!( + "{id} is over the ceiling on enforced rules and is advisory: a person signs \ + this one rule off with signed_off_by, or a rule is retired" + ), + policy: id, + kind: "over-ceiling".into(), + })); + let used: HashMap = self + .policy_rates() + .into_iter() + .map(|r| (r.id.clone(), r)) + .collect(); + let mut unused: Vec<&str> = p + .entries() + .iter() + .filter(|e| e.enforced() && p.state_of(e) == Status::Enforced) + .map(|e| e.meta.id.as_str()) + .filter(|id| used.get(*id).is_none_or(|r| r.refused == 0)) + .collect(); + unused.sort_unstable(); + unused.dedup(); + findings.extend(unused.into_iter().map(|id| Finding { + policy: id.to_string(), + kind: "unused".into(), + why: format!( + "{id} is enforced and refused nothing in the decision log: its owner retires \ + it, or says why it stays" + ), + })); + findings + } + + /// File a task for the owner for each finding of the lint, once per finding. A + /// rule that is not yet verified is no finding to act on, and gets none. Returns + /// the tasks filed now. + /// + /// # Errors + /// + /// A ledger error, or several granted domains and none to file in. + pub async fn file_policy_findings(&self, findings: &[Finding]) -> Result, Error> { + let domain = self.resolve_domain(None)?; + let mut filed = Vec::new(); + for f in findings.iter().filter(|f| f.kind != "inert") { + let created = self + .file_own(TaskSpec { + domain: domain.clone(), + title: format!("Policy bug, {}: {}", f.kind, f.policy), + body: format!( + "{}\n\nAcross layers the higher wins and the lower is fixed: amend or supersede {} so it agrees, or file an exception with a scope, an expiry and an approver. Within a layer, resolve from the stated intents and record the trade-off.", + f.why, f.policy + ), + requirements: vec!["human".into()], + origin: Some(format!("policy:{}", f.kind)), + upstream_event: Some(f.policy.clone()), + classification: dm_core::Label::Internal, + priority: 4, + ..TaskSpec::default() + }) + .await?; + if !created.deduplicated { + filed.push(created.task.id); + } + } + Ok(filed) + } } diff --git a/local/README.md b/local/README.md index 8b4032fe..2b3faa29 100644 --- a/local/README.md +++ b/local/README.md @@ -547,20 +547,24 @@ Thus an agent can propose a rule, but the rule is in no decision until a person it. `amem` loads no module from an entry below verified. A step from advisory to enforced is a new version of the entry, with `status = "enforced"`. The new version supersedes the old version, so the rule is proposed until a person verifies the new -version. `amem` evaluates the module of an advisory rule with each decision. It writes what the module would refuse to the decision -log (`advisories`), and it refuses nothing for that rule. `amem` refuses a module outside -the rules package, or a module that does not load. It names that module and leaves it -out, and the base policy stays in force. +version. `amem` evaluates the module of an advisory rule with each decision, apart from +the modules of the enforced rules. It writes what the module would refuse to the decision +log (`advisories`). An advisory module refuses nothing, whatever rule its denial names. +`amem` refuses a module outside the rules package, or a module that does not load. It +names that module and leaves it out, and the base policy stays in force. A session loads the policy when it starts, and again when it changes a policy entry itself. A change that a different session makes applies to the sessions that start after the change. `[gates] max_enforced_rules` in `grants.toml` is a ceiling on the enforced rules of one -domain. There is no ceiling by default. `amem` counts the rules of a domain in the order -of their ids. A rule over the ceiling is advisory. A rule with `signed_off_by` in its -block is enforced and is not counted, so a sign-off admits one rule and does not raise -the ceiling. +domain. There is no ceiling by default. `amem` counts the enforced rules of a domain from +the oldest entry to the newest. A rule over the ceiling is advisory, so a new rule does +not push out an older rule. `amem` does not count a rule in the L0 core, and the ceiling +never holds such a rule back. `amem` also does not count a verified rule with +`signed_off_by` in its block, when that name is a person. Thus a sign-off admits one rule +and does not raise the ceiling. The person who verifies that version of the rule gives +the sign-off its force. An L0 rule is in the core unless its block says `core = false`. A rule in the core takes no exception. @@ -574,28 +578,38 @@ Three memory tools use the rules. Each tool is also a command: - `check_policy` (`amem policy check `) gives the rules in force for a domain: the enforced rules and the advisory rules of layers 0 to 2, and the domain's own L3 rules. Each rule has its id, layer, rule, intent and status. `amem` does not select - the rules by the action. The read log records that the session asked. + the rules by the action. The read log records that the session asked. A session + reads a rule as it reads an entry. For a rule above the handling of the session, the + answer gives the id, layer and status, and `withheld`, and not the text. - `file_exception` (`amem policy except --task --scope --expiry - --justification`) writes an exception and proposes it for verified. The answer names - the approver: the owner of the rule for layers 2 and 3, and the owner of the record - for layers 0 and 1 and for a rule with no owner. `amem approve` on the task of the answer puts the exception in - force. `amem` does not check that the person who approves is the approver that the - answer names. + --justification`) writes an exception in the domain of the rule and proposes it for + verified. The scope and the justification are words. `amem` refuses a request with a + code fence in them. The answer names the approver: the owner of the rule for layers 2 + and 3, and the owner of the record for layers 0 and 1 and for a rule with no owner. + `amem approve` on the task of the answer puts the exception in force. A person who + did not file the exception approves it. `amem` does not check that this person is + the approver that the answer names. - `propose_amendment` (`amem policy amend --evidence`) files a task for a - person. The task names the owner of the rule and carries the evidence. The same - evidence for the same rule is one task. + person. The task names the owner of the rule and carries the evidence. The task has + the classification of the rule's entry. One filer has one open amendment for one rule: + a second call gives the same task. + +When two entries have the same policy id, these tools use the entry that is in force. `amem policy reset` retires the rules of a domain that have no person in their lineage. Only the owner of the record can run it. A rule has a person in its lineage when a -person wrote its entry, promoted it, or approved its promotion. The reset keeps those -rules. For each other rule, the reset writes a new version with `status = "retired"` -that supersedes the old version. The reset does not change exceptions. +person wrote its entry, promoted it to curated, or approved its promotion to verified. A +proposal that nobody approved does not count. The reset keeps those rules. For each +other rule, the reset writes a new version with `status = "retired"` that supersedes the +old version. The reset does not change exceptions or the rules of other domains. + +An exception with `status = "retired"` in its block is not in force. A denial from a record rule wins over the built-in decision, unless an exception covers the task that the session holds. Each decision names the policies that it rests on: the policies that refused it, or the exceptions that it used. Thus the decision log gives the -citations and the overrides of each policy. T0 carries the policy digest, L0 in full, the -verified rules by id, and the exceptions in force. +citations and the overrides of each policy. T0 carries the policy digest: the verified +rules of the L0 core in full, the other rules in force by id, and the exceptions in force. `amem policy list` shows the layer, trust tier and status of each policy. `amem policy lint` applies the paper's conflict rules. It finds: @@ -606,9 +620,8 @@ lint` applies the paper's conflict rules. It finds: - a rule that is not yet verified; - an enforced or advisory rule with no owner (`no-owner`); - an enforced or advisory rule after its `review_by` date (`review-due`); -- a rule whose status is lower than its block says (`held-back`); - a rule that the ceiling holds at advisory (`over-ceiling`); -- an enforced rule that no decision in the decision log cites (`unused`). +- an enforced rule with a module that refused nothing in the decision log (`unused`). It also gives, for each rule, how often it refused, how often an exception let a write through, and how often the rule advised. diff --git a/scripts/demo/3-perimeter.sh b/scripts/demo/3-perimeter.sh index acd303da..cd7bb899 100755 --- a/scripts/demo/3-perimeter.sh +++ b/scripts/demo/3-perimeter.sh @@ -93,7 +93,7 @@ say "A reset" "Rules are cheapest to add when something has just gone wrong. Whe agent agent:a write "{\"title\": \"pol-L3-900 No writes on a Friday\", \"entry_type\": \"policy\", \"body\": \"# No writes on a Friday\n\n\`\`\`policy\nid = \\\"pol-L3-900\\\"\nlayer = 3\nrule = \\\"No entry is written on a Friday.\\\"\nintent = \\\"A write on a Friday went wrong once.\\\"\nstatus = \\\"enforced\\\"\n\`\`\`\n\"}" >/dev/null agent agent:curator promote "{\"id\": \"$(a policy list | j "[p['entry'] for p in d['policies'] if p['id'] == 'pol-L3-900'][0]")\", \"to\": \"curated\", \"why\": \"it parses\"}" >/dev/null rules -a policy reset --domain dom-demo | j "'\n'.join([f\" kept {k['policy_id']}: {k['person']} is in its lineage\" for k in d['kept']] + [f\" retired {r}\" for r in d['retired']])" +a policy reset --domain dom-demo | j "'\n'.join([f\" kept {k['policy_id']}: {k['person']} is in its lineage\" for k in d['kept']] + [f\" retired {r['policy_id']}\" for r in d['retired']])" rules say "The lint" "Each rule and exception with how often it refused, let through and advised, counted from the decision log, so a rule nobody hits or an exception everybody uses shows up. A rule past its review date, a rule without an owner and an enforced rule that no decision cites are findings for the owner." From 9d0d6d639bd049625e0dc6693b212c2b3d0b09dd Mon Sep 17 00:00:00 2001 From: Sander Striker Date: Thu, 12 Nov 2026 15:09:58 +0100 Subject: [PATCH 3/4] fix: close what the second pass found in the rule lifecycle - A denial that is not an object, or whose rule is not a string, made each decision fail. The decision now reads each denial and each cited id as text. - All advisory modules ran in one engine, so one module that failed took each other report with it. Each advisory module now has its own engine. - The version of the policy now says which modules refuse and which report. - T0 gave the text of a rule above the handling of the session. The digest now gives only the id of such a rule. - `check_policy` and `propose_amendment` raised the label of the session from entries and tasks that they did not give. The label now rises only to the rules given. - An entry that nobody verified could speak for a rule when no entry was in force. Such a rule is now not found. - The bound of one amendment for each rule and filer now holds while a person works on the first task. - The words of an exception are one line with no markup. - `with_status` now changes the policy block where it is, and checks that the result reads back. - The reset counts an approver only when the entry is verified. Co-Authored-By: Claude Fable 5.2 --- PLAN.md | 2 +- README.md | 3 +- crates/policy/src/decide.rs | 194 ++++++++++++++++++++------- crates/policy/src/record.rs | 25 +++- crates/server/src/lib.rs | 260 ++++++++++++++++++++++++++++-------- crates/server/src/rules.rs | 86 +++++++----- local/README.md | 27 ++-- 7 files changed, 449 insertions(+), 148 deletions(-) diff --git a/PLAN.md b/PLAN.md index 1615628e..0ee4db37 100644 --- a/PLAN.md +++ b/PLAN.md @@ -70,7 +70,7 @@ Landed so far: | 1 | #71 | Nothing | | 2 | #73 | An attempt key on every operation; an uncertain external effect blocking reassignment; waiting on an outside event as a state; the record's remaining fields (`parent`, `scores`, `implements`, `reversibility`, `artifacts_touched`, `policy_context`) | | 3 | #74 | A suite that gates a version of a definition; the data scope enforced; a definition required by default, once the system's own filings name one; tools offered a few at boot and the rest by search | -| 6 | this pull request | `check_policy` selecting rules by the action; the unused-rule check over one review period; the approver of an exception checked against the layer breached; a new version of a rule keeping its place under the ceiling; a session that reloads the policy when a rule changes; a verified rule that stays in force while its next version waits for review | +| 6 | this pull request | `check_policy` selecting rules by the action; the unused-rule check over one review period; the approver of an exception checked against the layer breached; a new version of a rule keeping its place under the ceiling; a decision that survives an enforced module that fails; the author and the date of an imported rule not trusted; a session that reloads the policy when a rule changes; a verified rule that stays in force while its next version waits for review | What stays unbuilt after these, and stays on README's list: federation across domains, authentication of people, logprob-based drift, and accelerator time, which only a gateway can count. So do the items on that list that need another machine or an outside service more than they need code: the limits on re-enactment, harness settings and decomposition templates in the platform domain, bytes stored and resource limits on the sandbox, the authoring of agent definitions, the search tool against its live provider, a harness under Seatbelt on a Mac, and the Gofannon bridge against a live deployment, whose agents make model calls that the egress does not meter. diff --git a/README.md b/README.md index 335e2b24..c1c7bec1 100644 --- a/README.md +++ b/README.md @@ -129,7 +129,8 @@ two weeks. It is also the reason that none of it is the production shape. period. The approver of an exception is a name in the answer. `amem` does not check that this person approves. A new version of a rule is the newest rule under the ceiling. While the next version of a rule waits for its review, the rule is in no - decision. + decision. When the module of an enforced rule fails in a decision, that decision + fails. An imported rule keeps the author and the date of its entry. - **Lineage, in part.** An entry does not record its session. `amem taint` compares the time of a read with the time of a later write, and it goes forward only. Reads and sends are not in the event log. A write that the scan refuses is a file under `state/`, diff --git a/crates/policy/src/decide.rs b/crates/policy/src/decide.rs index e8683341..19784937 100644 --- a/crates/policy/src/decide.rs +++ b/crates/policy/src/decide.rs @@ -243,14 +243,18 @@ pub struct Decision { pub reason: String, /// The record's policies this decision rests on: those that refused it, or the /// exceptions it used. - #[serde(default)] + #[serde(default, deserialize_with = "as_texts")] pub cites: Vec, /// The digest of the policy text that decided. #[serde(default)] pub policy: String, /// Each rule that refused, with what a caller needs to act on the refusal: the /// rule, why, a reason code, and what would pass. - #[serde(default, skip_serializing_if = "Vec::is_empty")] + #[serde( + default, + deserialize_with = "as_denials", + skip_serializing_if = "Vec::is_empty" + )] pub denials: Vec, /// What the advisory rules would have refused, had they been enforced. They refuse /// nothing: this is how a rule is watched before it is given teeth. @@ -293,6 +297,35 @@ fn text_of(v: serde_json::Value) -> String { } } +/// A denial out of whatever a module put in its `deny` set: an object's fields, or +/// for anything else its JSON as the reason. +fn denial_of(v: serde_json::Value) -> Denial { + match v { + serde_json::Value::Object(_) => serde_json::from_value(v).unwrap_or_else(|e| Denial { + policy: String::new(), + reason: e.to_string(), + code: None, + remedy: None, + }), + other => Denial { + policy: String::new(), + reason: text_of(other), + code: None, + remedy: None, + }, + } +} + +fn as_denials<'de, D: serde::Deserializer<'de>>(d: D) -> Result, D::Error> { + let list = Vec::::deserialize(d)?; + Ok(list.into_iter().map(denial_of).collect()) +} + +fn as_texts<'de, D: serde::Deserializer<'de>>(d: D) -> Result, D::Error> { + let list = Vec::::deserialize(d)?; + Ok(list.into_iter().map(text_of).collect()) +} + fn as_text<'de, D: serde::Deserializer<'de>>(d: D) -> Result { serde_json::Value::deserialize(d).map(text_of) } @@ -400,9 +433,10 @@ pub struct Policy { refused: Vec<(String, String)>, /// The entries of the rules the ceiling holds at advisory. held_back: Vec, - /// The base with the modules of the advisory rules alone, when there are any. What - /// they deny is reported and never joins the decision. - watch: Option, + /// One engine for each advisory rule: the base with that rule's module alone. What + /// it denies is reported and never joins the decision, and a module that fails + /// takes no other rule's report with it. + watch: Vec<(String, regorus::Engine)>, } impl std::fmt::Debug for Policy { @@ -480,8 +514,8 @@ impl Policy { // an advisory rule's module is evaluated with every decision, in an engine of // its own: what it denies is reported, and whatever rule it names it cannot // refuse - let mut watch = self.engine.clone(); - let mut watched = false; + let base = self.engine.clone(); + let mut watch = Vec::new(); for e in entries.iter().filter(|e| e.enforced() || e.advisory()) { let Some(rego) = &e.rego else { continue }; if !crate::record::module_is_confined(rego) { @@ -491,19 +525,21 @@ impl Policy { } let name = format!("{}.rego", e.entry_id); let mut trial = if advisory(e) { - watch.clone() + base.clone() } else { self.engine.clone() }; + // the version says which modules are loaded and what each one does: the + // same module refuses as an enforced rule and reports as an advisory one + let how = if advisory(e) { "advisory" } else { "enforced" }; match trial.add_policy(name, rego.clone()) { - Ok(_) if advisory(e) => { - watch = trial; - watched = true; - texts.push_str(rego); - } Ok(_) => { - self.engine = trial; - texts.push_str(rego); + texts.push_str(&format!("\n# {} {how}\n{rego}", e.meta.id)); + if advisory(e) { + watch.push((e.meta.id.clone(), trial)); + } else { + self.engine = trial; + } } Err(err) => self.refused.push((e.meta.id.clone(), err.to_string())), } @@ -532,11 +568,10 @@ impl Policy { "grants": grants, "policies": policies, "exceptions": exceptions, }); self.engine = Policy::with_data(self.engine, &data)?; - self.watch = if watched { - Some(Policy::with_data(watch, &data)?) - } else { - None - }; + self.watch = watch + .into_iter() + .map(|(id, engine)| Ok((id, Policy::with_data(engine, &data)?))) + .collect::>()?; self.version = digest(&texts); self.entries = entries; self.held_back = over; @@ -569,41 +604,47 @@ impl Policy { ids } - /// What the advisory rules deny for a request. An advisory module that fails to - /// evaluate reports that, and changes no decision. + /// What the advisory rules deny for a request, each module asked on its own. A + /// module that fails to evaluate reports that under its rule, and changes no + /// decision and no other rule's report. A denial that names no rule is its + /// module's rule's. fn advisories(&self, input: &Input) -> Vec { - let Some(watch) = &self.watch else { + let Ok(input) = serde_json::to_string(input) else { return Vec::new(); }; - let failed = |why: String| { - vec![Denial { - policy: String::new(), - reason: format!("an advisory module did not evaluate: {why}"), - code: Some("advisory-failed".into()), - remedy: None, - }] - }; - let mut engine = watch.clone(); - let value = match serde_json::to_string(input) - .map_err(|e| e.to_string()) - .and_then(|j| regorus::Value::from_json_str(&j).map_err(|e| e.to_string())) - { - Ok(v) => v, - Err(why) => return failed(why), - }; - engine.set_input(value); - let said = engine - .eval_rule("data.amem.rules.deny".to_string()) - .map_err(|e| e.to_string()) - .and_then(|v| v.to_json_str().map_err(|e| e.to_string())) - .and_then(|j| serde_json::from_str::>(&j).map_err(|e| e.to_string())); - match said { - Ok(mut list) => { - list.sort_by(|a, b| (&a.policy, &a.reason).cmp(&(&b.policy, &b.reason))); - list + let mut out = Vec::new(); + for (id, watch) in &self.watch { + let mut engine = watch.clone(); + let said = regorus::Value::from_json_str(&input) + .map_err(|e| e.to_string()) + .and_then(|value| { + engine.set_input(value); + engine + .eval_rule("data.amem.rules.deny".to_string()) + .map_err(|e| e.to_string()) + }) + .and_then(|v| v.to_json_str().map_err(|e| e.to_string())) + .and_then(|j| { + serde_json::from_str::>(&j).map_err(|e| e.to_string()) + }); + match said { + Ok(list) => out.extend(list.into_iter().map(|v| { + let mut d = denial_of(v); + if d.policy.is_empty() { + d.policy.clone_from(id); + } + d + })), + Err(why) => out.push(Denial { + policy: id.clone(), + reason: format!("the advisory module did not evaluate: {why}"), + code: Some("advisory-failed".into()), + remedy: None, + }), } - Err(why) => failed(why), } + out.sort_by(|a, b| (&a.policy, &a.reason).cmp(&(&b.policy, &b.reason))); + out } fn with_data( @@ -652,7 +693,7 @@ impl Policy { entries: Vec::new(), refused: Vec::new(), held_back: Vec::new(), - watch: None, + watch: Vec::new(), }) } @@ -1271,6 +1312,57 @@ mod tests { ); let d = supersede_decision(&with(&g, vec![watch]), None); assert!(d.allow && d.advisories.len() == 1, "{d:?}"); + // a denial whose rule is not a string, and one that is no object at all + for said in [ + "{\"policy\": {\"a\": 1}, \"reason\": 3}", + "\"x\"", + "7", + "[1, 2]", + ] { + let module = format!("deny contains {said} if true"); + let d = supersede_decision( + &with(&g, vec![rule_with("pol-L2-061", "enforced", &module)]), + None, + ); + assert!(!d.allow, "{said}: {d:?}"); + assert_eq!(d.denials.len(), 1, "{said}: {d:?}"); + } + } + + /// Each advisory module is asked on its own: one that fails, or that has no + /// `deny`, reports that and takes no other rule's report with it. + #[test] + fn a_bad_advisory_module_takes_no_other_report_with_it() { + let g = grants(); + let good = rule_with( + "pol-L2-070", + "advisory", + "deny contains {\"policy\": \"pol-L2-070\", \"reason\": \"always\"} if true", + ); + for bad in ["deny contains \"x\" if true", "deny := 5", "allow := true"] { + let p = with( + &g, + vec![good.clone(), rule_with("pol-L2-071", "advisory", bad)], + ); + let d = supersede_decision(&p, None); + assert!(d.allow, "{bad}: {d:?}"); + assert!( + d.advisories + .iter() + .any(|a| a.policy == "pol-L2-070" && a.reason == "always"), + "{bad}: {d:?}" + ); + assert!( + d.advisories.iter().any(|a| a.policy == "pol-L2-071"), + "{bad}: {d:?}" + ); + } + // the version says what each module does + let module = "deny contains {\"reason\": \"always\"} if true"; + let advisory = with(&g, vec![rule_with("pol-L2-072", "advisory", module)]); + let enforced = with(&g, vec![rule_with("pol-L2-072", "enforced", module)]); + assert_ne!(advisory.version(), enforced.version()); + assert_ne!(advisory.version(), Policy::builtin(&g).unwrap().version()); } /// An exception whose block says retired is in force for nothing. diff --git a/crates/policy/src/record.rs b/crates/policy/src/record.rs index c734cc66..ba44433d 100644 --- a/crates/policy/src/record.rs +++ b/crates/policy/src/record.rs @@ -294,7 +294,18 @@ pub fn with_status(body: &str, status: Status) -> Result { let mut table: toml::Table = toml::from_str(block).map_err(|e| format!("policy block: {e}"))?; table.insert("status".into(), toml::Value::String(status.as_str().into())); let new = toml::to_string(&table).map_err(|e| format!("policy block: {e}"))?; - Ok(body.replacen(block, new.trim_end(), 1)) + // the block itself, by where it is: the same text can stand earlier in the body + let at = block.as_ptr() as usize - body.as_ptr() as usize; + let out = format!( + "{}{}{}", + &body[..at], + new.trim_end(), + &body[at + block.len()..] + ); + match parse("", dm_core::TrustTier::Raw, &out) { + Ok(e) if e.meta.status == Some(status) => Ok(out), + _ => Err("the policy block does not read back with the new status".into()), + } } /// The body of a policy entry: a heading, what a person reads, and the block. @@ -492,6 +503,18 @@ mod tests { }; assert!(v.enforced()); assert!(parse("x", TrustTier::Raw, "no block").is_err()); + // a new status goes into the block, wherever else its text stands in the body + let block = "id = \"pol-L2-031\"\nlayer = 2"; + let twice = format!("```toml\n{block}\n```\n\n```policy\n{block}\n```\n"); + let retired = with_status(&twice, Status::Retired).unwrap(); + assert_eq!( + parse("x", TrustTier::Raw, &retired).unwrap().meta.status, + Some(Status::Retired) + ); + assert!( + retired.starts_with(&format!("```toml\n{block}\n```")), + "{retired}" + ); assert!( parse("x", TrustTier::Raw, &body("layer = 2", None)).is_err(), "no id" diff --git a/crates/server/src/lib.rs b/crates/server/src/lib.rs index 46e8336e..29455407 100644 --- a/crates/server/src/lib.rs +++ b/crates/server/src/lib.rs @@ -1953,7 +1953,14 @@ impl Memory { self.observe(&decisions); self.observe(&conventions); let generated_by = format!("memory {}", env!("CARGO_PKG_VERSION")); - let digest = self.policy_digest(); + // a rule is read as an entry is: the digest gives the text of the rules this + // session may read, and for the others their ids + let readable: Vec = { + let f = self.session_filter()?; + let ix = self.index.lock().await; + ix.list(&f, Some(EntryType::Policy), 1000)? + }; + let digest = self.policy_digest(&readable); let text = t0::render(&t0::Inputs { domain: &domain, generated_by: &generated_by, @@ -1972,8 +1979,10 @@ impl Memory { /// The policy digest T0 carries: the L0 core in full, rule and intent; the other /// layers' rules that are verified, by id and rule, marked enforced where a module /// checks them; and the exceptions in force, with their scope. - fn policy_digest(&self) -> Vec { + fn policy_digest(&self, readable: &[Hit]) -> Vec { let p = self.policy(); + let withheld = |e: &dm_policy::record::Entry| !readable.iter().any(|h| h.id == e.entry_id); + let mut shown = Vec::new(); let now = OffsetDateTime::now_utc(); let mut out = Vec::new(); let mut entries: Vec<_> = p.entries().iter().collect(); @@ -1991,10 +2000,18 @@ impl Memory { )); } } else if m.is_core() && p.state_of(e) == dm_policy::record::Status::Enforced { - out.push(format!( - "{} (L0, not overridable): {} Why: {}", - m.id, m.rule, m.intent - )); + if withheld(e) { + out.push(format!( + "{} (L0, not overridable): above what this session may read", + m.id + )); + } else { + shown.push(e.entry_id.as_str()); + out.push(format!( + "{} (L0, not overridable): {} Why: {}", + m.id, m.rule, m.intent + )); + } } else if !m.is_core() { // a rule a person verified, or one that reports: with what its module // does, when it has one @@ -2009,10 +2026,23 @@ impl Memory { dm_policy::record::Status::Enforced | dm_policy::record::Status::Advisory ); if live && (e.tier == TrustTier::Verified || !how.is_empty()) { - out.push(format!("{} (L{}{how}): {}", m.id, m.layer, m.rule)); + let rule = if withheld(e) { + "above what this session may read" + } else { + shown.push(e.entry_id.as_str()); + m.rule.as_str() + }; + out.push(format!("{} (L{}{how}): {rule}", m.id, m.layer)); } } } + // the session has read the rules whose text it was given + let read: Vec = readable + .iter() + .filter(|h| shown.contains(&h.id.as_str())) + .cloned() + .collect(); + self.observe(&read); out } @@ -6621,6 +6651,69 @@ mod tests { assert!(matches!(unknown, Err(Error::NotFound(_)))); } + /// Two rules of an agent that a person proposed or closed and nobody approved: one + /// whose proposal was declined, and one whose proposal task a person completed + /// without approving, so the entry stayed raw. + async fn proposals_nobody_approved(root: &Path, owner: &Memory) { + let agent = Memory::open(root).unwrap().with_author("agent:a"); + let curator = Memory::open(root).unwrap().with_author("agent:curator"); + let declined = policy_entry(&agent, &rule_text("pol-L3-005", 3, "")).await; + let other = Memory::open(root).unwrap().with_author("human:test:other"); + let Promotion::Proposed { task } = other + .promote(&declined, TrustTier::Verified, "looks right") + .await + .unwrap() + else { + panic!("proposed") + }; + owner.invalidate(&task.id, "declined", None).await.unwrap(); + // and one whose proposal a person closed without approving: the entry is raw + let closed = policy_entry(&agent, &rule_text("pol-L3-006", 3, "")).await; + let Promotion::Proposed { task } = curator + .promote(&closed, TrustTier::Verified, "looks right") + .await + .unwrap() + else { + panic!("proposed") + }; + { + let mut l = owner.ledger.lock().await; + let who = "human:test:other"; + let lease = Duration::minutes(5); + l.claim( + &task.id, + who, + lease, + &task.spec.requirements, + Label::Restricted, + ) + .await + .unwrap(); + let held = dm_ledger::Held { + holder: who, + generation: 1, + }; + l.complete(&task.id, held, &closed).await.unwrap(); + } + } + + async fn retired_says_so_and_the_exception_stands(root: &Path, exception: &str) { + use dm_policy::record::Status as Rule; + let dir = root; + // the retired version says retired, and the exception is as it was + let fresh = Memory::open(dir).unwrap(); + let p = fresh.policy(); + let state = |id: &str| { + p.entries() + .iter() + .find(|e| e.meta.id == id) + .map(dm_policy::record::Entry::state) + }; + assert_eq!(state("pol-L3-003"), Some(Rule::Retired)); + assert_eq!(state("pol-L3-004"), Some(Rule::Retired)); + assert!(fresh.get(exception).await.unwrap().front.status.is_none()); + } + /// A reset keeps each rule that a person wrote, curated or approved and retires the /// rest. Only the owner resets. A proposal that a person made and nobody approved /// puts no person in the lineage, and an exception is not touched. @@ -6645,18 +6738,7 @@ mod tests { .await .unwrap(); policy_entry(&agent, &rule_text("pol-L3-004", 3, "")).await; - let declined = policy_entry(&agent, &rule_text("pol-L3-005", 3, "")).await; - let other = Memory::open(dir.path()) - .unwrap() - .with_author("human:test:other"); - let Promotion::Proposed { task } = other - .promote(&declined, TrustTier::Verified, "looks right") - .await - .unwrap() - else { - panic!("proposed") - }; - owner.invalidate(&task.id, "declined", None).await.unwrap(); + proposals_nobody_approved(dir.path(), &owner).await; let exception = "# Exception\n\n```policy\nid = \"exc-2026-030\"\nlayer = 3\nbreaches = \"pol-L3-002\"\nscope_task = \"task-me-x\"\nexpires = \"2099-01-01T00:00:00Z\"\n```\n"; let exception = policy_entry(&agent, exception).await; // a value that an agent wrote and nobody reviewed is not in the digest as core @@ -6676,7 +6758,13 @@ mod tests { let retired: Vec<_> = reset.retired.iter().map(|r| r.policy_id.as_str()).collect(); assert_eq!( retired, - ["pol-L0-102", "pol-L3-003", "pol-L3-004", "pol-L3-005"] + [ + "pol-L0-102", + "pol-L3-003", + "pol-L3-004", + "pol-L3-005", + "pol-L3-006" + ] ); let digest = Memory::open(dir.path()).unwrap().t0(None).await.unwrap(); assert!(!digest.contains("pol-L0-102"), "{digest}"); @@ -6686,18 +6774,7 @@ mod tests { status_of(dir.path(), "pol-L3-002").await, Some(Rule::Enforced) ); - // the retired version says retired, and the exception is as it was - let fresh = Memory::open(dir.path()).unwrap(); - let p = fresh.policy(); - let state = |id: &str| { - p.entries() - .iter() - .find(|e| e.meta.id == id) - .map(dm_policy::record::Entry::state) - }; - assert_eq!(state("pol-L3-003"), Some(Rule::Retired)); - assert_eq!(state("pol-L3-004"), Some(Rule::Retired)); - assert!(fresh.get(&exception).await.unwrap().front.status.is_none()); + retired_says_so_and_the_exception_stands(dir.path(), &exception).await; let again = owner.reset_rules(None).await.unwrap(); assert!(again.retired.is_empty(), "{:?}", again.retired); } @@ -6718,6 +6795,8 @@ mod tests { ("one task", smuggled), (smuggled, "Needed."), ("~~~policy", "x"), + ("one task\nbreaches = \"pol-L1-081\"", "x"), + ("one task", "Needed.