Prototype: Evaluate alternatives to YAJL for JSON processing

[Easton97-Jens]

Description

This pull request introduces a prototype exploring the potential replacement of YAJL or the use of alternative JSON libraries.

See related discussion in the issue:
👉 #3308

The goal is to evaluate the current JSON handling and investigate possible improvements in terms of:

• maintainability
• performance
• security
• modernization of dependencies

As part of this prototype, the following JSON libraries are being considered:

• RAPIDJSON
• NLOHMANNJSON
• JSONC (json-c)
• JANSSON
• CJSON
• JSONCPP
• JSONCONS
• SIMDJSON
• YYJSON
• GLAZE

This implementation should be considered experimental and primarily serves as a starting point for discussion.

Notes

• This is not a final solution, but an initial draft
• API/behavior may change
• Focus is on feasibility and early insights

Feedback

I’m explicitly open to:

• suggestions for improvement
• preferred libraries from the list
• compatibility constraints within the project
• performance/security considerations
• general feedback on the direction

Thanks for taking the time to review 🙌

[sonarqubecloud]

Quality Gate failed

Failed conditions
7 Security Hotspots
3.3% Duplication on New Code (required ≤ 3%)
E Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

[airween]

@Easton97-Jens,

first of all, thanks for this PR.

Unfortunately for some reason (probably because you completely re-organized the test cases) no regression tests were ran. That's very bad.

The other think is that I don't think we need as many different JSON parser as you've added here.

Here is my idea about this topic (add more JSON parser):

• under Drop YAJL dependency #3308 I mentioned the jsonbench tool
• I think we should extend that tool with more JSON parsers
• important: we should implement the possible solutions for both engine (v2 and v3) and make the tool available to choose the necessary one (for a user)
• then we (not just you and me, but the community) should start testing them thoroughly with so many different input
• we need to gather experience, like
  • performance
  • availability (how can we ensure that users can use any parser? As an operation system package? through a git submodule?)
  • knowledge, like
    • stream parsing (SAX) - this is mandatory
    • DOM parsing - this is necessary for tests, but we can replace it
    • this is a new expectation, but it might be I mentioned somewhere else: If the parser could perform schema validation, we could create a new operator: @validateJsonSchema

Based on these, we could choose 2 or 3 new JSON parsers, and that would be enough to add as a solution.

I know this is a hard work and not trivial, but I'm sure this would be a very big step forward in both engines.

[Easton97-Jens]

Thanks for your feedback. I'm working on the second prototype where I'm implementing two JSON modules: jsoncons and simdjson, as they're more closely related to Sax. Rapidjson is omitted because the last release is too old. You can find it at https://github.com/Easton97-Jens/ModSecurity branch v3/master-json2 see. It's not finished yet.

[airween]

Thanks for your feedback. I'm working on the second prototype where I'm implementing two JSON modules: jsoncons and simdjson, as they're more closely related to Sax. Rapidjson is omitted because the last release is too old. You can find it at https://github.com/Easton97-Jens/ModSecurity branch v3/master-json2 see. It's not finished yet.

Thanks.

Could you add them first to jsonbench?

[Easton97-Jens]

I opened a PR
digitalwave/jsonbench#1