Skip to content

fix(extract,cypher): decorators dropped after a comment; composite properties truncated at the first comma#1075

Open
JhohanBustamante wants to merge 2 commits into
DeusData:mainfrom
JhohanBustamante:fix/decorator-extraction-and-composite-props
Open

fix(extract,cypher): decorators dropped after a comment; composite properties truncated at the first comma#1075
JhohanBustamante wants to merge 2 commits into
DeusData:mainfrom
JhohanBustamante:fix/decorator-extraction-and-composite-props

Conversation

@JhohanBustamante

Copy link
Copy Markdown

Two independent bugs found while indexing a NestJS + Angular monorepo. Both make
decorator/route/authz queries unusable on frameworks whose decorators take
arguments — and one of them makes a well-documented endpoint disappear from the
graph entirely. Each commit is self-contained and carries a regression test.

1. A comment between decorators drops every decorator above it

extract_decorators() walks prev-siblings and breaks on the first NAMED node.
Comments are named nodes in tree-sitter, so a comment interleaved in a
decorator run ends the walk:

@Post('login')                     // <-- dropped
@HttpCode(HttpStatus.OK)           // <-- dropped
// throttled per IP and per account
@Throttle({ default: { ttl: 900_000, limit: 5 } })   // <-- kept
async login(dto: LoginDto) { ... }

The route then has no @Post, so it vanishes from any decorator-based query:
documenting a decorator removes the endpoint from the graph. On the backend I was
indexing, exactly 1 of 95 HTTP endpoints was missing — the one whose throttle
policy carried an explanatory comment.

Comments are now transparent to the walk, the same way the anonymous tokens
(e.g. TS export) already were. Reuses the existing is_comment_node() helper.

2. Composite property values are truncated at the first internal comma

json_extract_prop() scanned a non-string value up to the first , and a string
value up to the first ", ignoring nesting and backslash escapes. Any array or
object property was therefore cut at its first internal comma:

decorators: ["@Roles('OWNER', 'ADMIN')","@Get()"]
projected as: ["@Roles('OWNER'

So MATCH (m:Method) RETURN m.decorators returns garbage for NestJS, Angular or
Spring handlers (any decorator with more than one argument), and strings holding
an escaped quote are cut short too. Values are now copied as balanced constructs,
honoring string state and escape pairs; the scalar and plain-string paths are
untouched.

Combined effect on the repo that surfaced this: HTTP endpoints reachable from the
graph went from 3/95 to 95/95, with full @Roles(...) + verb + path.

Tests

  • tests/test_cypher.c: cypher_exec_prop_array_with_internal_commas,
    cypher_exec_prop_string_with_escaped_quote
  • tests/test_extraction.c: extract_ts_decorators_survive_interleaved_comment

test-runner cypher → 152 passed · test-runner extraction → 214 passed
(ASan + UBSan build). The full make -f Makefile.cbm test run got 4635 tests in
with no failures before it was OOM-killed on my 6 GB machine at the
parallel-index test (sanitizer memory, environmental — not a test failure), so CI
coverage there would be appreciated.

Comments are NAMED tree-sitter nodes, so the prev-sibling walk in
extract_decorators() stopped at one — silently dropping every decorator ABOVE
an interleaved comment:

    @post('login')                 <-- dropped
    @httpcode(HttpStatus.OK)       <-- dropped
    // throttled per IP and account
    @Throttle({ ... })             <-- kept
    async login(...)

The route then vanished from decorator/route queries, so documenting a
decorator made the endpoint disappear from the graph. Real-world impact: on a
NestJS backend, 1 of 95 HTTP endpoints was missing — the one whose throttle
policy carried an explanatory comment.

Comments are now transparent to the walk, the same way anonymous tokens
(e.g. TS `export`) already were. Reuses the existing is_comment_node() helper.

Signed-off-by: KolisCode <jhohantma@gmail.com>
json_extract_prop() scanned a non-string property up to the first ',' and a
string property up to the first '"', ignoring nesting and backslash escapes.
Any array/object property was therefore truncated at its first INTERNAL comma,
and any string containing an escaped quote was cut short.

    decorators: ["@roles('OWNER', 'ADMIN')","@get()"]
    projected as: ["@roles('OWNER'

This makes decorator/route/authz queries unusable on frameworks whose
decorators take multiple arguments (NestJS, Angular, Spring). Values are now
copied as balanced constructs, honoring string state and escape pairs; scalar
and plain-string paths are unchanged.

Signed-off-by: KolisCode <jhohantma@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant