Advice on how to use libfsm for generating performant pattern matchers

[gsusanto-fastly]

This PR introduces a new documentation file (doc/advice.md) that provides practical advice on using libfsm for generating reasonably performant pattern matching code.

[katef]

this isn't about doing only what's specified, it's about writing what we mean. without the anchors, you'd allow a prefix and a suffix. if you mean that, great. if you don't mean that, we're advising to not write them

[gsusanto-fastly]

I updated by adding "When the intention is to match an entire string, use anchors."
Let me know if you still think it is better to not write them.

[katef]

it's worth explaining why this happens

[gsusanto-fastly]

Added some explanation and link back to the Writing Effective Regex guide. Let me know if you think it is too repetitive or too long.

[katef]

looks great, thank you

[silentbicycle]

This is such a significant feature that it may be worth saying a little more here?

Do you think the target audience will know what backtracking is, or the implications of that? Maybe saying that it doesn't need to make any complex decisions at runtime? In contrast to PCRE, where the same regex can have wildly different runtime costs depending on the input.

[gsusanto-fastly]

I added some information, what do you think?

[katef]

I think if we have anything else to say about that, it should go in the top-level readme. we're underselling there

[silentbicycle]

Group capture is coming soon :)

[gsusanto-fastly]

I added "coming soon" in the places I mention "group capture".
Let me know if I should retract those.

[katef]

that's fine!

[silentbicycle]

Depending on what's being repeated, bounded repeats can be substantially more expensive, because instead of representing and then adding a loop around it, the FSM needs to explicitly represent 50 instances of the pattern (which, in this case, are all optional) in order to have the fixed upper limit.

[katef]

definitely worth explaining

[silentbicycle]

Perhaps a more informative example: If you have the pattern ^x{3,5}$, libfsm's resulting DFA will be structured like "match an x, then match an x, then match an x, then match an x or skip it, then match an x or skip it, then report an overall match if at the end of input". It has to repeat the pattern, noting each time whether it's required or optional (beyond the lower count in {min,max}), because DFA execution doesn't have a counter, just the current state within the overall DFA.

[gsusanto-fastly]

Thank you for the explanation!
Now that I know that the suggestion is not correct, I have removed it from the docs.

Do you think this information is good to be included in the document?
If so, in which section should it be added into? Is it recommended to not use this? What is the alternative instead?

Or is it just a "good to know" section? What other "good to know" do you think is worth mentioning in the document?

[katef]

I'm going to add back in a section about bounded repetition

[gsusanto-fastly]

Thank you so much for the reviews!
I have updated the docs based on the reviews, further comments are more than welcomed.

[katef]

it's not "risky", it's either incorrect, or it's unnecessary complexity

[gsusanto-fastly]

I changed it to:

# Incorrect: allows a trailing newline,
# which is usually unintended and adds unnecessary complexity

Let me know if this is a bit wordy?