parser: replace internal frontmatter lex with rustc_lexer::tokenize

[dimxy]

Currently RA has its own shebang and frontmatter tokenizing code while the rustc tokenizer also does this job.
This PR eliminates internal frontmatter lex code and uses rustc_lexer::tokenize instead.
There are important notes about this code though:

• Rustc performs its frontmatter validation in the parser (not in the tokenizer) so this PR has to borrow and implement those checks from the rustc parser.
• Unlike RA, rustc_lexer::tokenize receives only normalized input (CRLF -> LF), so when used inside RA, it may return '\r' suffix in the frontmatter token and treat some infostrings with ending '\r' as invalid. Both these cases are processed in the PR's validate_frontmatter code. A test was added for CRLF input as well.

Fixes #23144. Added a test case.

Disclaimer: used Claude Code to help with code analysis and reviewing changes.