SQLite engine: multi-byte UTF-8 in a query comment silently truncates the generated query string

[mwtrigg]

Version

sqlc v1.31.1 (also reproduced against whatever latest resolved to on 2026-07-23, installed via go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest).

What happened

With the sqlite engine, if a SQL comment immediately preceding a -- name: query annotation contains a multi-byte UTF-8 character (an em dash —, a section sign §, etc.), the query string sqlc embeds in the generated Go code gets silently truncated at the end — cutting off part of the last clause. This produces Go code that compiles fine (it's a syntactically valid string literal) but fails at runtime with a SQL syntax error or "no such column" error, since the emitted SQL text is not what was written.

In one real case in a larger project this also produced a stray leading ; before an UPDATE statement in the generated const, in addition to truncating its trailing RETURNING clause.

The truncation length appears to scale with the extra byte-vs-rune width of the non-ASCII character(s) preceding the query in the file — consistent with an offset computed once as a rune count and applied once as a byte count (or vice versa) when slicing the original source text to produce the embedded query string.

Minimal reproduction

schema.sql:

CREATE TABLE repos (
    id       INTEGER PRIMARY KEY,
    name     TEXT NOT NULL,
    cap_read TEXT NOT NULL
);

queries/repos.sql:

-- name: ListAnonymousReadRepos :many
-- barn-spec.md notes -- an em dash right here — is enough to trigger it.
SELECT id, name, cap_read
FROM repos WHERE cap_read = 'anonymous' ORDER BY name;

sqlc.yaml:

version: "2"
sql:
  - engine: "sqlite"
    schema: "schema.sql"
    queries: "queries"
    gen:
      go:
        package: "db"
        out: "db"

Run sqlc generate. The generated db/repos.sql.go contains:

const listAnonymousReadRepos = `-- name: ListAnonymousReadRepos :many
SELECT id, name, cap_read
FROM repos WHERE cap_read = 'anonymous' ORDER BY na
`

Note ORDER BY na — the trailing me of name is gone. sqlc generate itself reports no error or warning; the only sign anything is wrong is in the emitted string.

Expected behavior

The generated query string should always be a byte-for-byte-correct copy of the original SQL statement, regardless of what non-ASCII characters appear in preceding comments. At minimum, sqlc generate/sqlc vet should fail loudly rather than emit a truncated, syntactically-different query silently.

Impact

This is a correctness bug with no warning signal short of manually diffing every generated query against its source, or (as we ended up doing) noticing a runtime SQL error and tracing it back. For a tool whose entire value proposition is "catch SQL mistakes at generate time instead of runtime," a silently-corrupted-but-still-compiling query is close to a worst case. Removing all non-ASCII characters from queries/*.sql comments fixed it for us, and we've added a CI tripwire (grep -P '[^\x00-\x7F]' over the queries directory) so it doesn't recur, but obviously that's a project-side workaround, not a fix.

Happy to provide more detail or test additional inputs if useful for narrowing down the exact offset-calculation bug.

[muratmirgun]

It looks like the same issue as #4235 and #4372: the SQLite parser gets rune-based offsets from ANTLR, but those offsets are later used to slice a string by bytes.

I also found #4314 and #4373, which already have fixes for this. Are either of those PRs still active? If not, I’d be happy to pick this up and send an updated fix.