Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 59 additions & 26 deletions PROOF-NEEDS.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -107,12 +107,43 @@ definitions. A mutant that blanks with `+/+` instead of a space is rejected.
Nine closed sanity theorems, including two comments on different lines and
`+///+`.

The model does *not* match `strip_proof_comments`, so it is evidence about
the model only. It replaces comment characters with spaces and keeps every
newline. The Rust function deletes comment text and drops a trailing
newline: on `//x` the model gives three spaces and Rust gives the empty
string. The model also covers line comments only; block comments and string
literals are not modelled.
The model now agrees with `strip_proof_comments(s, "//", None)`, the slice
it models. The compiled model and the Rust function give the same output on
every string of length at most 9 over `/`, newline, `x` and space (349,525
inputs, local run). In CI,
`the_line_comment_slice_matches_the_idris_model` re-checks a Rust
transcription of the model's equations up to length 8. Correspondence on
that slice is therefore `TESTED`, with those bounds, not `ASSUMED`. Block
comments, nesting and string literals are not modelled. Before the
single-pass rewrite they disagreed: on `//x` the model gave three spaces and
Rust gave the empty string.

|`strip_proof_comments` blanks comments in place. The output has the input's
byte length; every byte outside a comment, and every newline, is unchanged;
other comment bytes become spaces. Stripping is idempotent. Whichever
comment opens first wins, and block comments nest.
|`src/assail/analyzer.rs`, module `strip_proof_comments_tests`
|`TESTED`
|Exhaustive over bounded inputs, for the marker sets the analyzers pass:
`//` with `/* */`, `--` with `/- -/`, `--` with `{- -}`, `(* *)` alone, and
`//` alone. Each runs every string over a 4-letter alphabet up to length 8
(87,381 inputs) or a 5-letter alphabet up to length 7 (97,656 inputs). Each
alphabet spells the markers, plus a newline and an ordinary character.
The three counterexamples in the open obligations are regression tests, as
are nesting, unterminated blocks and multibyte text.

Planted faults, all caught:

* the previous two-phase function;
* deleting comment text instead of blanking it;
* blanking that drops a newline;
* one space per character instead of per byte;
* no nesting;
* an unterminated block that keeps code.

One mutant, a line comment that also consumes its newline, is equivalent:
the newline is kept and the scan resumes after it, so no test can tell
them apart.

|Attestation chain: integrity, authenticity, non-repudiation
|`src/abi/AttestationUnforgeability.idr`
Expand Down Expand Up @@ -223,30 +254,32 @@ generation, and keep the `keep` newest older ones
|*OPEN*. The model check above covers the protocol, not the Rust code;
these contracts tie the implementation to P1 and P2.

|`strip_proof_comments` is idempotent and preserves positions outside
comments and strings
|Fix the implementation first; then Creusot/Pearlite and Verus
|Neither property holds of the current Rust function, so neither can be
proved until it changes. All three cases below were run against the
function as it is on `main`:
|Prove the `strip_proof_comments` properties in the ledger for all inputs
|Creusot/Pearlite and Verus
|The single-pass rewrite makes both properties true of the implementation;
the ledger row has them `TESTED` within bounds. Pearlite contracts can
state them directly: equal length, the per-byte relation, and
idempotence. Three counterexamples held of the previous two-phase function
and are now regression tests:

* Not idempotent: `a//*x*/*b*/c` gives `a/*b*/c`, and a second pass gives
* Not idempotent: `a//*x*/*b*/c` gave `a/*b*/c`, and a second pass gave
`ac`.
* Positions move: `a/*x*/b` gives `ab`, moving `b` from offset 6 to 1.
* Ordering bug: block comments are stripped before line comments, so a
block opener inside a line comment swallows the code after it.
`code // see /* note`, then `hidden();`, then `*/ tail`, gives `code `.
The callers that pass both a line and a block marker are affected:
* Positions moved: `a/*x*/b` gave `ab`, moving `b` from offset 6 to 1.
* Ordering bug: block comments were stripped before line comments, so a
block opener inside a line comment swallowed the code after it.
`code // see /* note`, then `hidden();`, then `*/ tail`, gave `code `.
The callers that pass both a line and a block marker were affected:
`analyze_rust`, `is_rust_ffi_safe_wrapper`, `analyze_idris`,
`analyze_lean` and `analyze_agda`. In the Idris, Lean and Agda ProofDrift
detectors such a comment can hide a later `sorry`, `believe_me` or
postulate. `analyze_isabelle` and `analyze_coq` pass no line marker and
are not affected.

The Idris2 model above can guide the contracts once the function is
fixed; it does not discharge them. The model's own remaining slices
(block comments, strings, composition, position preservation; issue #114)
are optional once the implementation proof exists.
detectors such a comment could hide a later `sorry`, `believe_me` or
postulate.

The function does not know about string literals. The Rust and Zig callers
blank strings first; the Lean, Agda, Idris, Coq and Isabelle callers do
not, so a comment marker inside a string literal there still starts a
comment. The Idris2 model's remaining slices (block comments, strings,
composition; issue #114) are optional once the implementation proof
exists.

|Patch Bridge reachability is sound: no reachable dependency is classified
as phantom
Expand Down
8 changes: 8 additions & 0 deletions src/abi/Stripping.idr
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@
||| The model strips ALL `//` comments: `stripLineCommentBody` hands back
||| to `stripLineComments` after each preserved newline (issue #113).
|||
||| Correspondence: the model is `strip_proof_comments(s, "//", None)`, the
||| line-comment slice only. Block comments, their nesting and string
||| literals are not modelled, so the theorem is about that slice. On it the
||| compiled model and the Rust function agree on every string of length at
||| most 9 over `/`, newline, `x` and space, and a Rust test
||| (`the_line_comment_slice_matches_the_idris_model`) re-checks these
||| equations up to length 8. See PROOF-NEEDS.adoc.
|||
||| Re-proved 2026-11-09. The previous proof (PR #111, issue #113) never
||| type-checked: its `//` case relied on reductions that do not hold for a
||| variable character, and it used `decEq` without importing it. The proof
Expand Down
256 changes: 221 additions & 35 deletions src/assail/analyzer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6187,51 +6187,63 @@ fn push_line_anchored_proof_drift(
});
}
}

/// Remove proof-language comments while keeping every newline, so a line
/// number taken from the result is a line number in `content`.
/// Blank out proof-language comments in one left-to-right pass.
///
/// Every byte inside a comment becomes a space, except `'\n'`, which is
/// kept. So the result has exactly the length of `content`, every byte
/// outside a comment is unchanged, and a line or column taken from the
/// result is a line or column in `content`.
///
/// Whichever comment opens first wins: a block opener inside a line comment
/// is comment text, and so is a line marker inside a block comment. Block
/// comments nest, as they do in every caller's language (Rust, Lean, Agda,
/// Idris, Coq/Rocq and Isabelle). An unterminated block comment runs to the
/// end of `content`. An empty `line_marker`, or an empty block marker,
/// switches that kind of comment off. Markers must not contain whitespace.
///
/// The result is a fixed point: stripping it again changes nothing.
fn strip_proof_comments(content: &str, line_marker: &str, block: Option<(&str, &str)>) -> String {
// First strip block comments greedily, left-to-right.
let mut without_blocks = String::with_capacity(content.len());
if let Some((open, close)) = block {
let mut rest = content;
while let Some(start) = rest.find(open) {
without_blocks.push_str(&rest[..start]);
let after_open = &rest[start + open.len()..];
if let Some(end) = after_open.find(close) {
push_newlines_of(&mut without_blocks, &after_open[..end]);
rest = &after_open[end + close.len()..];
fn blank(out: &mut String, span: &str) {
for c in span.chars() {
if c == '\n' {
out.push('\n');
} else {
// Unterminated block comment — drop the rest, keep its lines.
push_newlines_of(&mut without_blocks, after_open);
rest = "";
break;
// One space per byte keeps byte offsets, not just columns.
out.extend(std::iter::repeat_n(' ', c.len_utf8()));
}
}
without_blocks.push_str(rest);
} else {
without_blocks.push_str(content);
}

// Then strip line comments. Split on '\n' (not `lines()`) so a trailing
// newline survives and the newline count is exactly preserved.
if line_marker.is_empty() {
return without_blocks;
}
let mut out = String::with_capacity(without_blocks.len());
for (i, line) in without_blocks.split('\n').enumerate() {
let kept = match line.find(line_marker) {
Some(idx) => &line[..idx],
None => line,
let block = block.filter(|(open, close)| !open.is_empty() && !close.is_empty());
let mut out = String::with_capacity(content.len());
let mut depth = 0usize;
let mut i = 0;
while let Some(rest) = content.get(i..).filter(|r| !r.is_empty()) {
let step = match block {
Some((_, close)) if depth > 0 && rest.starts_with(close) => {
depth -= 1;
close.len()
}
Some((open, _)) if rest.starts_with(open) => {
depth += 1;
open.len()
}
_ if depth > 0 => rest.chars().next().map_or(1, char::len_utf8),
_ if !line_marker.is_empty() && rest.starts_with(line_marker) => {
rest.find('\n').unwrap_or(rest.len())
}
_ => {
let len = rest.chars().next().map_or(1, char::len_utf8);
out.push_str(&rest[..len]);
i += len;
continue;
}
};
if i > 0 {
out.push('\n');
}
out.push_str(kept);
blank(&mut out, &rest[..step]);
i += step;
}
out
}

/// Strip C-family line comments (`//`, `///`, `//!`) from each line, leaving
/// the rest of the source intact. String literals are detected so a `//`
/// inside `"http://example.com"` is NOT treated as a comment.
Expand Down Expand Up @@ -8885,3 +8897,177 @@ mod proof_line_anchor_tests {
assert_eq!(occurrence_lines(&code, "sorry"), vec![6]);
}
}

#[cfg(test)]
mod strip_proof_comments_tests {
use super::strip_proof_comments;

type Markers = (&'static str, Option<(&'static str, &'static str)>);

/// The marker pairs the analyzers pass, each with an alphabet that can
/// spell every marker, plus a newline and an ordinary character.
const CONFIGS: [(Markers, &[char]); 5] = [
(("//", Some(("/*", "*/"))), &['/', '*', '\n', 'x']),
(("--", Some(("/-", "-/"))), &['/', '-', '\n', 'x']),
(("--", Some(("{-", "-}"))), &['{', '-', '}', '\n', 'x']),
(("", Some(("(*", "*)"))), &['(', '*', ')', '\n', 'x']),
(("//", None), &['/', '\n', 'x', ' ']),
];

/// Every string over `alphabet` of length at most `max_len`.
fn all_strings(alphabet: &[char], max_len: usize) -> Vec<String> {
let mut all = vec![String::new()];
let mut layer = vec![String::new()];
for _ in 0..max_len {
layer = layer
.iter()
.flat_map(|s| {
alphabet.iter().map(move |c| {
let mut t = s.clone();
t.push(*c);
t
})
})
.collect();
all.extend(layer.iter().cloned());
}
all
}

fn strip(s: &str, (line, block): Markers) -> String {
strip_proof_comments(s, line, block)
}

#[test]
fn a_block_opener_inside_a_line_comment_hides_nothing() {
let src = "code // see /* note\nhidden();\n*/ tail";
let out = strip(src, ("//", Some(("/*", "*/"))));
let blanked = " ".repeat("// see /* note".len());
assert_eq!(out, format!("code {blanked}\nhidden();\n*/ tail"));
}

#[test]
fn a_line_marker_inside_a_block_comment_hides_nothing() {
let src = "/- -- -/ sorry";
assert_eq!(strip(src, ("--", Some(("/-", "-/")))), " sorry");
}

#[test]
fn the_former_idempotence_counterexample_is_now_a_fixed_point() {
let markers = ("//", Some(("/*", "*/")));
let once = strip("a//*x*/*b*/c", markers);
assert_eq!(once, "a ");
assert_eq!(strip(&once, markers), once);
}

#[test]
fn text_after_a_block_comment_keeps_its_offset() {
let out = strip("a/*x*/b", ("//", Some(("/*", "*/"))));
assert_eq!(out, "a b");
assert_eq!(out.find('b'), Some(6));
}

#[test]
fn block_comments_nest() {
let src = "/- a /- b -/ sorry -/ x";
assert_eq!(
strip(src, ("--", Some(("/-", "-/")))),
" x"
);
let src = "(* (* *) sorry *) oops";
assert_eq!(
strip(src, ("", Some(("(*", "*)")))),
" oops"
);
}

#[test]
fn an_unterminated_block_comment_runs_to_the_end_keeping_newlines() {
let out = strip("a {- b\nc\n", ("--", Some(("{-", "-}"))));
assert_eq!(out, "a \n \n");
}

#[test]
fn multibyte_text_keeps_its_byte_offsets() {
let src = "/* é */x // ü\ny";
let out = strip(src, ("//", Some(("/*", "*/"))));
assert_eq!(out.len(), src.len());
assert_eq!(out.find('x'), src.find('x'));
assert_eq!(out.find('y'), src.find('y'));
assert!(!out.contains('é') && !out.contains('ü'));
}

#[test]
fn empty_markers_switch_that_comment_kind_off() {
assert_eq!(strip("a // b", ("", None)), "a // b");
assert_eq!(strip("a /* b */", ("", Some(("", "*/")))), "a /* b */");
}

/// Exhaustive over every string up to the bound, for every marker set the
/// analyzers use: the result has the input's length, keeps every byte
/// outside a comment and every newline, blanks the rest with spaces, and
/// is a fixed point.
#[test]
fn stripping_preserves_positions_and_is_idempotent_exhaustively() {
for (markers, alphabet) in CONFIGS {
let max_len = if alphabet.len() > 4 { 7 } else { 8 };
for src in all_strings(alphabet, max_len) {
let out = strip(&src, markers);
assert_eq!(out.len(), src.len(), "{markers:?} {src:?}");
for (a, b) in src.bytes().zip(out.bytes()) {
assert!(
a == b || (b == b' ' && a != b'\n'),
Comment thread
hyperpolymath marked this conversation as resolved.
Resolved
"{markers:?} {src:?} -> {out:?}"
);
}
assert_eq!(strip(&out, markers), out, "{markers:?} {src:?}");
}
}
}

/// The line-comment model in `src/abi/Stripping.idr`,
/// transcribed equation by equation. Idris2 proves the model idempotent.
fn idris_model(s: &[char]) -> String {
fn code(s: &[char], out: &mut String) {
match s {
[] => {}
['/', '/', rest @ ..] => {
out.push_str(" ");
body(rest, out);
}
[c, rest @ ..] => {
out.push(*c);
code(rest, out);
}
}
}
fn body(s: &[char], out: &mut String) {
match s {
[] => {}
['\n', rest @ ..] => {
out.push('\n');
code(rest, out);
}
[_, rest @ ..] => {
out.push(' ');
body(rest, out);
}
}
}
let mut out = String::new();
code(s, &mut out);
out
}

/// With only a `//` marker, the Rust function and the Idris2 model agree
/// on every input up to the bound, so the model's idempotence theorem is
/// about this function's behaviour on that slice, not about a different
/// function.
#[test]
fn the_line_comment_slice_matches_the_idris_model() {
for src in all_strings(&['/', '\n', 'x', ' '], 8) {
let chars: Vec<char> = src.chars().collect();
assert_eq!(strip(&src, ("//", None)), idris_model(&chars), "{src:?}");
}
}
}
Loading