[minleejae]

The generated parser now passes its grammar locations (the "line:col" strings used in error messages) as indexes into a table that is loaded with the first error message. On JSQLParser, the parser class goes from 47,463 to 32,603 of its 65,535 constant pool entries. Error messages are unchanged, and parsing is about 3% faster in JSQLParser's JMH benchmark.

This follows up on JSQLParser/JSqlParser#2866, where @manticore-projects agreed to a generator change. Issues are disabled here, so I'm sending the PR directly.

Why

A class file holds at most 65,535 constant pool entries. JSQLParser's CCJSqlParser is at 47,463 on current master (a30a679), up from 46,304 on 2026-10-28 and 12,266 in JSQLParser 5.3. JaCoCo already fails on it with ClassTooLargeException, so JSQLParser now excludes it from coverage (JSQLParser/JSqlParser#2867). javac fails once the parser itself reaches the limit.

Every jj_consume_token and jj_scan_token call and every jj_la1_loc entry passes its grammar location as a string literal. Each distinct location takes two constant pool entries, a String and its Utf8: in JSQLParser's parser, 7,436 locations take 14,872 entries, 31% of the pool. The locations are only read to build error messages.

In JSQLParser/JSqlParser#2866 I suggested moving the jj_3* routines into a nested class. I tried that first. It brings the parser class to 38,744 entries, but in JSQLParser's JMH benchmark parsing gets 5–6% slower (16.50 / 16.60 / 16.75 → 17.51 / 17.69 / 17.59 ms/op), as the lookahead routines then reach the parser's fields through the outer instance. Moving the locations out saves more and leaves the parsing code as it is.

Change (ParserCodeGenerator)

Generated code	jj_consume_token, jj_scan_token, jj_la1_loc and the _locs arrays of the token set choices take an int location: the 1-based index of the "line:col" string in a table, or 0 for none
Location table	jj_loc(int) returns the string from JJLocations.LOCATIONS, a nested class that builds the table from string constants of at most 60,000 characters. The class is loaded when the first error message is built
Error messages	generateParseException(String), jj_add_error_token and trace_expected get the strings through jj_loc, so the messages are unchanged
jj_consume_token(int, String)	still generated, for grammar code that calls it with a string (JSQLParser does)
Without ERROR_REPORTING	unchanged, as there are no locations

The generation of jj_consume_token moved into a method, emitConsumeToken, to generate both versions. Its body is unchanged.

Verification

On JSQLParser master a30a679:

Check	Result
Baseline: release built locally against javacc-core 8.1.3.1943	generates the same JSQLParser sources as the published javacc-java 8.1.3.1943
Generated parser	same as with release once the strings are put back in place of their indexes and the int jj_consume_token, jj_loc and JJLocations are removed, apart from the checksum line
Constant pool, parser class	47,463 → 32,603 (JJLocations: 43)
JaCoCo 0.8.14 instrumentation, parser class	ClassTooLargeException → 52,363 entries
gradlew check	12,918 tests pass
Parse results and error messages for 38,301 inputs: JSQLParser's 321 test scripts, and ~4,200 statements in 3 dialects, each as is and in 2 broken forms	same as master. 22,176 are parse errors, 21,129 of them listing expected tokens, including those from jj_rescan_token
JMH JSQLParserBenchmark (latest), 3 alternating runs of 3 forks × 5 iterations	master 17.22 / 16.75 / 16.69 ms/op, this branch 16.20 / 16.14 / 16.17 ms/op

Grammars from this repository's history (68 under examples/, grammars/ and issues/ at 028e00a, and issues/bas03 from #1):

Check	Result
10 option sets: default, STATIC, ERROR_REPORTING=false, CACHE_TOKENS, DEBUG_LOOKAHEAD, DEBUG_PARSER, DEPTH_LIMIT=100, JAVA_TEMPLATE_TYPE=modern, FORCE_LA_CHECK, LEGACY_EXCEPTION_HANDLING=false	in every set, all generated files are the same as with release once the locations are put back (481, or 679 with the modern template), and every parser that compiles with javac --release 8 from release also compiles from this branch
Java 1.5 grammar with ERROR_REPORTING=true, with and without STATIC, parsing JSQLParser's 1,244 source files	same results and error messages as release: 601 parsed, 643 errors with locations
issues/bas03 (token set choices, replayed by jj_rescan_token)	same output as release, and the error files match the expected ones

Not in this PR

• The nested class for the jj_3* routines, given its cost. With this change, JSQLParser's parser has room for a while without it.
• JSQLParser can stop excluding the parser from JaCoCo once it uses a release with this change.

AI assistance: prepared with Claude Code.

[minleejae]

Following up on the side question in JSQLParser/JSqlParser#2866: since JSQLParser's build depends on this fork and JavaCC.org is no longer responding, it might be worth hosting the fork under the JSQLParser organization. JSQLParser contributors who hit a generator limit, like the constant pool here, would find it more easily, and issues could be enabled there for reports like this one. I understand it also serves your other projects, so feel free to ignore this if it doesn't fit.

[manticore-projects]

Following up on the side question in JSQLParser/JSqlParser#2866: since JSQLParser's build depends on this fork and JavaCC.org is no longer responding, it might be worth hosting the fork under the JSQLParser organization. JSQLParser contributors who hit a generator limit, like the constant pool here, would find it more easily, and issues could be enabled there for reports like this one. I understand it also serves your other projects, so feel free to ignore this if it doesn't fit.

In a perfect world with reasonable people, where everyone cooperates smoothly for the best of human mankind, your were absolutely right.
In the unfortunate reality, I have been spending too much time on waiting and chasing for people. And at the same time, JSQLFormatter and MJDBCUtils are critical for us -- and so indirectly JSQLParser and JavaCC became critical as well. For that reason I will keep that fork until someone picks it up, up-streams it and maintains it reliably.

I hold no power over JSQLParser upstream, there was no stable release for almost 2 years for that reason.

[manticore-projects]

Of course your suggestion has one huge advantage: testing! JSQLParser is probably the most sophisticated consumer of JavaCC-8 and every change should be tested against the test suite and benchmarks of JSQLParser first.
If I had free hands, I would implement that as a Gradle Multiproject where JavaCC and JSQLParser and JSQLFormatter and MJDBCUtils and JSQLTranspiler are sub projects.

[minleejae]

Hi @manticore-projects Thanks for sharing your perspective. Your reasons for keeping the fork make sense, and I agree that testing JavaCC changes against JSQLParser’s test suite and benchmarks would be valuable.

[manticore-projects]

Published to SonaType Central and bumped in JSQLParser already.
The performance improvement is lovely:

Benchmark                               (version)  Mode  Cnt   Score   Error  Units
JSQLParserBenchmark.parseSQLStatements     latest  avgt   15  20.440 ± 0.219  ms/op
JSQLParserBenchmark.parseSQLStatements        5.4  avgt   15  32.334 ± 0.285  ms/op

Thank you so much!