Skip to content

refactor(sql): one prepared-VALUES splitter, one literal skipper, and no silent VALUE truncation - #480

Merged
MPCoreDeveloper merged 1 commit into
masterfrom
fix/prepared-values-dedup
Oct 26, 2026
Merged

MPCoreDeveloper merged 1 commit into
masterfrom
fix/prepared-values-dedup

Conversation

@MPCoreDeveloper

Copy link
Copy Markdown
Owner

Why

Sonar flagged new-code duplication in the 2.1.2 diff above the 3 % ceiling. The duplicated blocks were real: the prepared-insert path carried five copies of two algorithms, and the copies had already drifted apart in behaviour.

What changed — one algorithm per concern

Concern Single source of truth Copies removed
Quote/literal scanning SqlLiteralScanner the two inline scanners in Database.Batch.cs, behind a new TrySkipLiteral helper
VALUES splitting ValueRangeEnumerator (ref struct, yields [start,end)) the two identical loops in ParseValues / ParseValuesToArray
? placeholder positions ParameterBinder.ExtractPositionalParameterPositions a byte-for-byte copy in PreparedStatement

The enumerator is a ref struct on purpose: the prepared-insert path must not allocate per row, so the dedup costs no allocation. It keeps the old semantics exactly — single-quote literals skipped as a whole through SqlLiteralScanner, () and [] (VECTOR literals) suppressed, top-level comma splits, same bounds guards.

Defect the dedup exposed, and fixed

The prepared/batch path stopped counting a tuple at the table's column count:

CREATE TABLE docs (id INTEGER PRIMARY KEY, name TEXT, score REAL);
INSERT INTO docs VALUES (1, 'a', 0.5, 21);   -- 4 values, 3 columns

It bound the first three values, discarded 21 and succeeded. The statement path has rejected that shape since [2.1.2], so the two disagreed. Surplus values now throw (InvalidOperationException; the message names the table, the tuple and both counts), and a short tuple keeps its documented meaning (missing columns fall to NULL). This was the last copy of the known-issue-7 truncation class.

Tests

  • New PreparedInsertValuesParsingTests (11 cases): every case is parsed through both prepared parsers and asserted to agree — neither had any coverage before, which is how they could drift. Covers quoted commas, doubled quotes, backslashes, VECTOR literals, NULL, short tuples and the surplus-tuple rejection.
  • Three fixtures had been passing because of the truncation: FixedWidthBulkUpdateTests (five DDLs realigned to the 4-column docs shape its own helper inserts into), FormatCompatPolicyTests and DefaultEngineSelectionTests. The latter two also rendered i * 0.5 as 0,5 on this nl-NL machine — two SQL values — and now format invariant, as EncryptionCoverageTests and WritePathProfilerTests already do.

Verification

  • Core suite on this exact tree: 2118 / 0 failed / 0 skipped (Category!=Debug&Category!=Manual&Category!=Performance), build clean on net11.0.
  • Canaries green: ReopenRoundTripMatrixTests, FormatCompatPolicyTests, SingleFileDirectoryParityTests, FixedWidthBulkUpdate/Delete/PatchTests, WritePathProfilerTests, FixedWidthInlineValueTests.Reopen_KeepsInlineAndOverflowValues.
  • The §7 reproduction published in docs/sharpcoredb-known-issues.md still prints the documented fixed output for all five bound-value shapes (re-run live against this tree).
  • CHANGELOG.md gains Fixed and Changed entries under [Unreleased] — the strictness is user-visible.

No API change, no format change, no durability or encryption behaviour touched.

…t VALUE truncation

Sonar failed the 2.1.2 CI gate on new-code duplication (over the 3 % ceiling):
the prepared-insert path carried five copies of two algorithms. Fixed the cause
rather than suppressing the gate.

One shared algorithm per concern:

- PreparedInsertStatement.ParseValues and ParseValuesToArray held identical
  splitting loops. Both now drive one `ValueRangeEnumerator` ref struct that
  yields [start,end) bounds — allocation-free, same paren/bracket/quote
  handling, same bounds guards.
- The two canonical-shape scanners (IsInsertStatement, TryReadLiteralUntilWhere)
  re-implemented the quote machine. They now call SqlLiteralScanner through a
  shared TrySkipLiteral helper.
- PreparedStatement.ExtractParameterPositions was a byte-for-byte copy of
  ParameterBinder.ExtractPositionalParameterPositions; the copy is gone.

Defect exposed by the dedup, and fixed: the batch/prepared path stopped counting
a tuple at the table's column count, so `INSERT INTO docs VALUES (1,'a',0.5,21)`
against a three-column table discarded the 4th value and succeeded, while the
statement path has rejected that shape since 2.1.2. Surplus values now throw
with the table, tuple and both counts; a short tuple still fills NULLs.

Tests: new PreparedInsertValuesParsingTests (11 cases) parses every case through
both prepared parsers and asserts they agree — neither had coverage before.
Three test fixtures passed 4 values to 3-column tables and only passed because
of the truncation (FixedWidthBulkUpdateTests DDLs realigned with their own
4-column fixture; FormatCompatPolicyTests and DefaultEngineSelectionTests also
rendered `i * 0.5` as "0,5" on this nl-NL machine, i.e. two values — now
formatted invariant, as EncryptionCoverageTests already does).

Verified on this tree: core suite 2118 / 0 failed, build clean (net11.0),
and the §7 reproduction in docs/sharpcoredb-known-issues.md still prints the
documented fixed output for all five bound-value shapes.
@MPCoreDeveloper MPCoreDeveloper changed the title refactor(sql): one VALUES splitter and one literal skipper, and stop silently dropping surplus VALUES refactor(sql): one prepared-VALUES splitter, one literal skipper, and no silent VALUE truncation Oct 26, 2026
@sonarqubecloud

Copy link
Copy Markdown

@MPCoreDeveloper
MPCoreDeveloper merged commit 2f56114 into master Oct 26, 2026
13 checks passed
@codecov

codecov Bot commented Oct 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 77.41935% with 14 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...c/SharpCoreDB/Database/Execution/Database.Batch.cs 78.68% 7 Missing and 6 partials ⚠️
src/SharpCoreDB/Services/PreparedStatements.cs 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@MPCoreDeveloper
MPCoreDeveloper deleted the fix/prepared-values-dedup branch October 26, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant