Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Ramstack.Parsing documentation

Ramstack.Parsing is a parser combinator library for .NET. A parser reads text and
returns a value. You build larger parsers by joining small ones: a character,
a word, a number, a list, or a whole expression.

Start with [Getting started](getting-started.md). Then use the other pages as a
guide to the methods you need.

| Page | What it covers |
| --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| [Getting started](getting-started.md) | Install the library, run a parser, read results, handle spaces, and require the whole input. |
| [Basic parsers](basic-parsers.md) | `L`, `OneOf`, `Set`, Unicode categories, `Character`, `Any`, `S`, `Eol`, `Eof`, and `Regex`. |
| [Numbers and quoted text](literals.md) | `Literal.Number<T>`, `NumberKind`, quoted strings, quoted characters, and escape sequences. |
| [Combining parsers](combining-parsers.md) | `Seq`, `Then`, `ThenIgnore`, `Between`, `Choice`, `Or`, `Optional`, `DefaultOnFail`, `Return`, `And`, and `Not`. |
| [Repetition and lists](repetition-and-lists.md) | `ZeroOrMore`, `Many`, `OneOrMore`, `AtLeast`, `Repeat`, `Separated`, and `Until`. |
| [Working with values](working-with-values.md) | `Do`, `Where`, `Map`, `Text`, `Void`, `Match`, `Unit`, and when callbacks run. |
| [Recursive parsers and expressions](recursive-parsers.md) | `Deferred`, `Recursive`, `FoldL`, `FoldR`, `Prefixed`, and operator precedence. |
| [Errors](errors.md) | Error messages, `As`, `Error`, `Fail`, `FatalError`, and exceptions. |
| [Writing your own parser](custom-parsers.md) | `ParseContext` and a custom `Parser<T>` class. |
| [Complete examples](examples.md) | A list of integers, a setting, a calculator, and links to the project samples. |
| [Common mistakes](common-mistakes.md) | Problems that new users often meet, and how to fix them. |
| [Performance tips](performance-tips.md) | How to keep parsers fast. |

## Find a method by task

| I want to... | Use |
| ---------------------------------------- | ---------------------------------------------------------- |
| Read an exact character or word | `L` |
| Read one of several words | `OneOf` |
| Read one character from a range or a set | `Set`, `Character` |
| Read a number | `Literal.Number<T>` |
| Read a quoted string | `Literal.DoubleQuotedString`, `Literal.SingleQuotedString` |
| Skip spaces | `S` |
| Require the end of the input | `Eof` |
| Run parsers one after another | `Seq`, `Then`, `ThenIgnore`, `Between` |
| Try alternatives | `Choice`, `Or` |
| Make a part optional | `Optional`, `DefaultOnFail` |
| Repeat a parser | `ZeroOrMore`, `OneOrMore`, `Repeat`, `AtLeast` |
| Read a list with separators | `Separated` |
| Read until an end marker | `Until` |
| Change the value | `Do` |
| Get the source text | `Text` |
| Check a value | `Where` |
| Look ahead without reading | `And`, `Not` |
| Read operators such as `+` or `^` | `FoldL`, `FoldR` |
| Read prefix operators such as `-` or `!` | `Prefixed` |
| Let a parser use itself | `Deferred`, `Recursive` |
| Give a parser a better error name | `As` |

## Words used in these docs

- **Failure**: a parser does not match the text. This is normal. A choice tries
the next alternative, and a repetition stops.
- **Fatal error**: `Fail` or `FatalError` stops the whole parse. Nothing recovers from it.
- **Callback**: a function that you pass to a method, such as the one in `Do`.
- **Matched segment**: the part of the input that a parser matched.
`Text` and `Map` read it.
- **Void parser**: a parser that throws its value away. `Unit` is the empty value
that it returns.
- **Prefix**: the text at the start of the input.

## Reading the examples

The examples use these imports. Add them to a console app before you copy a
snippet. Each snippet is separate unless the page says otherwise.

```csharp
using System;
using System.Collections.Generic;
using System.Globalization;
using Ramstack.Parsing;
using static Ramstack.Parsing.Parser;
```

Comments after `Console.WriteLine` show the expected output. Small examples
often read only the start of the input, to show how a method works. The
complete examples use `Eof` to check the whole input.

`ParseResult<T>.Value` has a nullable type. When an example knows that the parse
succeeded, it writes `result.Value!`.

The library targets .NET 6, 7, 8, and 9. It also works in later .NET versions.
`Parser.Regex` needs .NET 7 or later. `NumberKind.BinaryNumber` needs .NET 8 or
later (.NET 9 for `BigInteger`). All other examples compile with C# 10, the
default for .NET 6.

## Common rules

- Parsing starts at the current position. A parser does not search ahead for a match.
- Spaces are part of the input. Add `S` or your own whitespace parser where needed.
- A successful parse can leave unread text. Add `ThenIgnore(Eof)` to reject it.
- A choice uses the first alternative that succeeds. Put longer and more specific
alternatives first.
- Repetition is greedy. It reads as much as it can and never gives text back.
- Keep the functions that build values free of side effects. They may run during
an attempt that later fails, or be skipped when the value is thrown away. See
[Callbacks and thrown-away values](working-with-values.md#callbacks-and-thrown-away-values).
- Build a parser once and reuse it. Finish setting up recursive parsers before use.

See [Common mistakes](common-mistakes.md) for examples of what goes wrong when you
forget these rules. See the [project README](../README.md) for a short overview
and benchmark links.
238 changes: 238 additions & 0 deletions docs/basic-parsers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,238 @@
# Basic parsers

[Documentation index](README.md)

These parsers match text at the current position. They do not skip spaces or
require the end of the input unless you add those parsers. See the index for the
[imports used in examples](README.md#reading-the-examples).

## L: one character or one literal string

`L(char)` reads one exact character. `L(string, comparison)` reads an exact
string. Both return the literal they matched.

```csharp
Console.WriteLine(L(':').Parse(":rest").Value); // :
Console.WriteLine(L("hello").Parse("hello!").Value); // hello
Console.WriteLine(L("hello").Parse("Hello").Success); // False
```

String matching uses `StringComparison.Ordinal` by default. Pass a comparison
to change it:

```csharp
var word = L("hello", StringComparison.OrdinalIgnoreCase);

Console.WriteLine(word.Parse("HELLO").Value); // hello
Console.WriteLine(word.Text().Parse("HELLO").Value); // HELLO
```

The returned value is the literal you gave to `L`, even when the source has
different letter case. Use `Text()` to get the source spelling.

Use ordinal comparisons (the default) for programming languages and file formats.
A culture-based comparison reads the current culture when you build the parser.
Changing the culture later does not change that parser. It can also match text
that is shorter or longer than the literal, and in rare cases it matches without
reading any text. Avoid it inside repeated parsers.

`L("")` is not allowed. Use `Return(value)` for a parser that reads nothing.

## OneOf: characters or words

`OneOf("+-")` reads one character from the string. It returns a `char`.
The string is a list of characters, so `OneOf("a-z")` means `a`, `-`, or `z`.
Use `Set("a-z")` for a range.

`OneOf("yes", "no")` or `OneOf(new[] { "yes", "no" })` reads one whole
string and returns a `string`. There are also overloads for three or four
strings, and the string overloads accept a `StringComparison`.

```csharp
Console.WriteLine(OneOf("+-").Parse("-1").Value); // -
Console.WriteLine(OneOf("yes", "no").Parse("yes!").Value); // yes

var comparison = OneOf("<", "<=", ">", ">=");
Console.WriteLine(comparison.Parse("<=").Value); // <=
```

With ordinal comparisons, `OneOf` tries longer words first. So `OneOf("<", "<=")`
reads `<=` when the input has it. The order in which you wrote the words does not
matter. [`Choice`](combining-parsers.md#choice-and-or-try-alternatives) is
different: it tries alternatives in the order you gave them.

With culture-based comparisons, `OneOf` tries the words in descending sort order.
Words that compare equal are merged, and only the first one is kept. This also
applies to ignore-case comparisons:

```csharp
var ab = OneOf("Ab", "ab", StringComparison.OrdinalIgnoreCase);

Console.WriteLine(ab.Parse("ab").Value); // Ab
Console.WriteLine(ab.Text().Parse("ab").Value); // ab
```

As with `L`, the result is the word you supplied. Use `Text()` to get the source
text.

The character string, the array of words, and each word must be non-empty.

## Set: a character range or class

`Set(lo, hi)` reads one character in an inclusive range. `Set(pattern)` reads
one character from a class written without square brackets.

```csharp
Console.WriteLine(Set('a', 'z').Parse("cat").Value); // c
Console.WriteLine(Set("A-Za-z0-9_").Parse("_id").Value); // _
Console.WriteLine(Set(@"\p{L}").Parse("é").Value); // é
Console.WriteLine(Set(@"\d").Parse("5").Value); // 5
```

| Pattern | Matches |
| ---------------------------------------------------- | ---------------------------------------------------------------------------- |
| `a-z`, `0-9` | An inclusive range. |
| `abc_` | Any of these literal characters. |
| `\p{L}`, `\p{Nd}` | A Unicode general category. |
| `\P{L}` | A character outside that Unicode general category. |
| `\d`, `\D` | A Unicode decimal digit, or a character outside that class. |
| `\w`, `\W` | A word character, or a character outside that class. |
| `\s`, `\S` | Whitespace, or a character outside that class. |
| `\n`, `\r`, `\t`, `\a`, `\b`, `\v`, `\f`, `\e`, `\0` | A character escape. Here `\b` means backspace and `\0` is the NUL character. |
| `\\`, `\-` | A literal backslash or hyphen. |
| `\u0041` | A character given by exactly four hexadecimal digits. |

Use a verbatim C# string, such as `@"\d"`, or double the backslash in a normal
string, such as `"\\d"`.

A hyphen makes a range only when it sits between two characters. At the start or
at the end of the pattern it is an ordinary hyphen, so `Set("+-")` reads `+` or
`-`. In the middle, write `\-`:

```csharp
Console.WriteLine(Set("+-").Parse("-").Success); // True
Console.WriteLine(Set(@"a\-z").Parse("-").Success); // True
Console.WriteLine(Set(@"a\-z").Parse("m").Success); // False
```

`Set` is a character-class parser, not a full regular expression. These
characters have no special meaning in it: `^`, `[`, `]`, and `.`. They match
themselves.

```csharp
Console.WriteLine(Set("^a").Parse("^").Success); // True
Console.WriteLine(Set("^a").Parse("b").Success); // False
Console.WriteLine(Set("[a-z]").Parse("[").Success); // True
```

So `Set("^a")` does not mean "any character except `a`". `Set` also has no class
subtraction, no anchors, and no named Unicode blocks such as `\p{IsGreek}`. A bad
pattern, such as `\q`, `\.`, or a backward range like `z-a`, throws
`ArgumentException` when you build the parser. To read a character other than a
comma, use `Not(L(',')).Then(Any)`.

## Unicode categories and Character

`L(UnicodeCategory)` reads a character from one .NET Unicode category.
`L(GeneralUnicodeCategory)` accepts flags, including groups such as `Letter`,
`Number`, and `Punctuation`.

```csharp
var letter = L(UnicodeCategory.LowercaseLetter);
var letterOrDigit = L(
GeneralUnicodeCategory.Letter | GeneralUnicodeCategory.DecimalDigitNumber);

Console.WriteLine(letter.Parse("é").Value); // é
Console.WriteLine(letterOrDigit.Parse("7").Value); // 7
```

`GeneralUnicodeCategory.None` is not a valid category for `L`. It throws
`ArgumentException`.

The `Character` class provides ready-made `Parser<char>` properties:

| Property | What it reads |
| ----------------------------------------------- | --------------------------------------------------- |
| `Letter` | Any Unicode letter. |
| `Uppercase`, `Lowercase` | An uppercase letter, or a lowercase letter. |
| `Digit` | A decimal digit from any script. |
| `Number` | A character in any Unicode number category. |
| `LetterOrDigit` | A letter or a decimal digit. |
| `HexDigit` | An ASCII hexadecimal digit: `0-9`, `A-F`, or `a-f`. |
| `Symbol`, `Separator`, `Punctuation`, `Control` | A character in the matching Unicode category group. |
| `WhiteSpace` | One whitespace character. |
| `Surrogate`, `HighSurrogate`, `LowSurrogate` | One UTF-16 surrogate code unit. |

```csharp
Console.WriteLine(Character.Letter.Parse("Ж").Value); // Ж
Console.WriteLine(Character.HexDigit.Parse("F").Value); // F
Console.WriteLine(Character.Digit.Parse("٣").Success); // True
Console.WriteLine(Set("0-9").Parse("٣").Success); // False
```

These parsers read one C# `char` at a time. Some characters, such as emoji, use
two `char` values. `Any` and the `Character` parsers do not join the pair.

## Any, S, Eol, and Eof

| Parser | Result | What it does |
| ------ | ------ | -------------------------------------------------------------------------------------- |
| `Any` | `char` | Reads any one UTF-16 code unit, including a line break. Fails at the end of the input. |
| `S` | `Unit` | Reads zero or more whitespace characters, using `char.IsWhiteSpace`. Always succeeds. |
| `Eol` | `Unit` | Reads a line break. At the end of the input it succeeds without reading anything. |
| `Eof` | `Unit` | Succeeds only at the end of the input, without reading anything. |

`Unit` means that the parser has no useful value to return.

```csharp
Console.WriteLine(Any.Parse("abc").Value); // a
Console.WriteLine(S.Parse(" \tvalue").Length); // 2
Console.WriteLine(Eol.Parse("\r\nnext").Length); // 2
Console.WriteLine(Eol.Parse("").Success); // True
Console.WriteLine(Eof.Parse("x").Success); // False
```

`Eol` reads CRLF as one line break. It also reads LF, CR, U+0085, U+2028,
and U+2029. Because it can succeed without reading anything at the end of the
input, do not use it alone in a loop that must move forward.

## Regex (.NET 7 or later)

`Regex(System.Text.RegularExpressions.Regex)` turns a .NET regular expression
into a `Parser<string>`. The pattern must start with the character `^`. Without it,
`Regex` throws `ArgumentException`. The match must start at the current position.

```csharp
var code = Regex(new System.Text.RegularExpressions.Regex("^[A-Z]{2}[0-9]{3}"))
.ThenIgnore(Eof);

Console.WriteLine(code.Parse("AB123").Value); // AB123
Console.WriteLine(code.Parse("xAB123").Success); // False
```

The parser returns the matched text. It fails when no text is left, even if the
pattern could match an empty string. When some text is left, a pattern such as
`^a*` can match zero characters. Then the parser succeeds and reads nothing. Use
`Eof` when the whole input must match.

The regular expression sees only the text that is left, not the text that was
already read. A lookbehind such as `(?<=a)b` cannot see earlier characters, and
`\b` at the start always sees a word boundary:

```csharp
var lookbehind = L('a').Then(Regex(new System.Text.RegularExpressions.Regex("^(?<=a)b")));
Console.WriteLine(lookbehind.Parse("ab").Success); // False
```

The error message uses the pattern text. Give the parser a name with `As` to
get a message that is easier to read:

```csharp
var code = Regex(new System.Text.RegularExpressions.Regex("^[A-Z]{2}[0-9]{3}"));
Console.WriteLine(code.Parse("1").ErrorMessage); // (1:1) Expected ^[A-Z]{2}[0-9]{3}

var named = code.As("product code");
Console.WriteLine(named.Parse("1").ErrorMessage); // (1:1) Expected product code
```

Next: [Numbers and quoted text](literals.md).
Loading
Loading