Skip to content

Fix syntax error positions being offset when comments precede brackets#72

Draft
Copilot wants to merge 2 commits intomainfrom
copilot/fix-syntax-error-position
Draft

Fix syntax error positions being offset when comments precede brackets#72
Copilot wants to merge 2 commits intomainfrom
copilot/fix-syntax-error-position

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 12, 2026

validate_syntax() used token-list indices for position reporting, but tokenize() strips comments first — so any non-token characters before a bracket caused the reported char position to be wrong.

Example: >comment[ would report char 2 (index in token list) instead of char 9 (original source position).

Changes

  • tokens.py — Added internal _tokenize_with_positions() that returns (Token, original_char_index) tuples, preserving each token's position in the original source string.
  • interpreter.pyvalidate_syntax() now accepts position-annotated tokens and uses original_char_index for error messages; run() uses _tokenize_with_positions() for both validation and token extraction.
  • tests/test_interpreter.py — Added parametrized cases with comments before brackets to assert correct original-source positions are reported.

Copilot AI changed the title [WIP] Fix comment removal before syntax validation Fix syntax error positions being offset when comments precede brackets Apr 12, 2026
Copilot AI requested a review from shenanigansd April 12, 2026 17:59
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.07%. Comparing base (65287d3) to head (0654aae).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #72      +/-   ##
==========================================
+ Coverage   99.04%   99.07%   +0.03%     
==========================================
  Files          10       10              
  Lines         209      217       +8     
  Branches       31       33       +2     
==========================================
+ Hits          207      215       +8     
  Partials        2        2              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Comments are removed before checking for syntax errors, which include position information

2 participants