Commit f3a0463
[Pratt Parser] Add a fuzzer comparing ANTLR and Pratt parser outputs and fix uncovered discrepancies
Added `CelPrattParserFuzzer` to fuzz CEL inputs against both `AntlrParser` and `PrattParser`, asserting error parity and AST equality.
Parser differences uncovered and fixed in `PrattParser` / `Lexer`:
1. **Mixed unary operator chains (e.g., `-!ll`, `!-x`)**: ANTLR requires consecutive unary operators to be homogeneous (`!`/`-`) and only allows `-` after `!` when immediately followed by an integer or floating-point literal (e.g., `!-42`), whereas Pratt previously allowed arbitrary mixtures of `!` and `-` without parentheses.
2. **Vertical tab (`\v`, ASCII 11)**: ANTLR does not treat `\v` as whitespace, whereas `Lexer` and `PrattParser` previously skipped it.
3. **Unquoted `.in` field selector**: ANTLR treats `in` as a keyword token and rejects unquoted `x.in` (requiring backtick-quoted `` x.`in` ``), whereas Pratt previously accepted unquoted `.in` after `.`.
4. **Chained optional select (`T.?a.?a`) AST positions**: ANTLR records the position of the field constant in `_?._` at the start of the `member` expression (`T`), whereas Pratt stopped at intermediate `.?`/`[]`/`()` nodes.
5. **Numeric literals immediately followed by identifier characters (e.g., `9in-x`)**: ANTLR tokenizes numeric literals (`NUM_INT`, `NUM_UINT`, `NUM_FLOAT`) without rejecting trailing identifier characters so `9in-x` parses as `9 in -x`, whereas Pratt's `Lexer` previously rejected trailing identifier characters at lexing time.
6. **Invalid quoted field selectors inside `has(...)` (e.g., `` has(a.`$b`) ``)**: When `normalizeIdent()` rejects an invalid backtick-quoted field name, `PrattParser` previously still constructed a `CelSelect` with an empty field string, causing `CelExprFactory.newSelect()` to throw `IllegalArgumentException` during `has()` macro expansion instead of returning an unset error expression like `AntlrParser`.
Intentional parser differences ignored by `CelPrattParserFuzzer` (where `PrattParser` behavior is preferred):
1. **Raw byte string literal prefixes (`rb'...'`, `rB'...'`, `Rb'...'`, `RB'...'`)**: ANTLR only accepts `br`/`bR`/`Br`/`BR` prefix order, whereas Pratt accepts both `br` and `rb`.
2. **Standalone commas in empty collection literals (`[,]`, `{,}`, `Msg{,}`)**: ANTLR accepts empty collections containing only a comma, whereas Pratt requires at least one element/entry before a trailing comma.
3. **Leading-dot identifier positions (`.R`)**: Pratt records the position of leading-dot identifiers at the `.` token, whereas ANTLR records it at the identifier token after `.`.
PiperOrigin-RevId: 9870078231 parent 5d344bc commit f3a0463
10 files changed
Lines changed: 718 additions & 129 deletions
File tree
- parser/src
- main/java/dev/cel/parser
- test
- java/dev/cel/parser
- resources
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
428 | 428 | | |
429 | 429 | | |
430 | 430 | | |
431 | | - | |
432 | 431 | | |
433 | 432 | | |
434 | 433 | | |
| |||
589 | 588 | | |
590 | 589 | | |
591 | 590 | | |
592 | | - | |
| 591 | + | |
593 | 592 | | |
594 | 593 | | |
595 | 594 | | |
596 | 595 | | |
597 | 596 | | |
598 | | - | |
599 | | - | |
600 | | - | |
601 | | - | |
602 | | - | |
603 | | - | |
604 | 597 | | |
605 | 598 | | |
606 | 599 | | |
| |||
622 | 615 | | |
623 | 616 | | |
624 | 617 | | |
625 | | - | |
626 | | - | |
627 | | - | |
628 | | - | |
629 | 618 | | |
630 | 619 | | |
631 | 620 | | |
| |||
0 commit comments