fix(extract,cypher): decorators dropped after a comment; composite properties truncated at the first comma#1075
Open
JhohanBustamante wants to merge 2 commits into
Conversation
Comments are NAMED tree-sitter nodes, so the prev-sibling walk in
extract_decorators() stopped at one — silently dropping every decorator ABOVE
an interleaved comment:
@post('login') <-- dropped
@httpcode(HttpStatus.OK) <-- dropped
// throttled per IP and account
@Throttle({ ... }) <-- kept
async login(...)
The route then vanished from decorator/route queries, so documenting a
decorator made the endpoint disappear from the graph. Real-world impact: on a
NestJS backend, 1 of 95 HTTP endpoints was missing — the one whose throttle
policy carried an explanatory comment.
Comments are now transparent to the walk, the same way anonymous tokens
(e.g. TS `export`) already were. Reuses the existing is_comment_node() helper.
Signed-off-by: KolisCode <jhohantma@gmail.com>
json_extract_prop() scanned a non-string property up to the first ',' and a
string property up to the first '"', ignoring nesting and backslash escapes.
Any array/object property was therefore truncated at its first INTERNAL comma,
and any string containing an escaped quote was cut short.
decorators: ["@roles('OWNER', 'ADMIN')","@get()"]
projected as: ["@roles('OWNER'
This makes decorator/route/authz queries unusable on frameworks whose
decorators take multiple arguments (NestJS, Angular, Spring). Values are now
copied as balanced constructs, honoring string state and escape pairs; scalar
and plain-string paths are unchanged.
Signed-off-by: KolisCode <jhohantma@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent bugs found while indexing a NestJS + Angular monorepo. Both make
decorator/route/authz queries unusable on frameworks whose decorators take
arguments — and one of them makes a well-documented endpoint disappear from the
graph entirely. Each commit is self-contained and carries a regression test.
1. A comment between decorators drops every decorator above it
extract_decorators()walks prev-siblings and breaks on the first NAMED node.Comments are named nodes in tree-sitter, so a comment interleaved in a
decorator run ends the walk:
The route then has no
@Post, so it vanishes from any decorator-based query:documenting a decorator removes the endpoint from the graph. On the backend I was
indexing, exactly 1 of 95 HTTP endpoints was missing — the one whose throttle
policy carried an explanatory comment.
Comments are now transparent to the walk, the same way the anonymous tokens
(e.g. TS
export) already were. Reuses the existingis_comment_node()helper.2. Composite property values are truncated at the first internal comma
json_extract_prop()scanned a non-string value up to the first,and a stringvalue up to the first
", ignoring nesting and backslash escapes. Any array orobject property was therefore cut at its first internal comma:
So
MATCH (m:Method) RETURN m.decoratorsreturns garbage for NestJS, Angular orSpring handlers (any decorator with more than one argument), and strings holding
an escaped quote are cut short too. Values are now copied as balanced constructs,
honoring string state and escape pairs; the scalar and plain-string paths are
untouched.
Combined effect on the repo that surfaced this: HTTP endpoints reachable from the
graph went from 3/95 to 95/95, with full
@Roles(...)+ verb + path.Tests
tests/test_cypher.c:cypher_exec_prop_array_with_internal_commas,cypher_exec_prop_string_with_escaped_quotetests/test_extraction.c:extract_ts_decorators_survive_interleaved_commenttest-runner cypher→ 152 passed ·test-runner extraction→ 214 passed(ASan + UBSan build). The full
make -f Makefile.cbm testrun got 4635 tests inwith no failures before it was OOM-killed on my 6 GB machine at the
parallel-index test (sanitizer memory, environmental — not a test failure), so CI
coverage there would be appreciated.