Fix SQL reference persistence and repeated statement scans

DEVELOPER_GUIDE.md

@@ -2349,6 +2349,22 @@ TypeScript decorators emit `annotation` rows for the decorator name and must not
 
 C# named-argument labels such as `overwrite:` are syntax, not type positions, and must not emit `type_reference` rows. The declaration-type scanner skips a leading single-colon label inside an argument fragment, including comma-terminated multiline argument lines, while preserving expression references, named `out` declaration types, explicitly typed lambda and anonymous-method parameters, and typed LINQ range variables in the argument value. Multiline property subpatterns likewise keep the type after their property label. Alias-qualified names (`Alias::Type`), statement and `case` labels, nullable types, and ternary expressions remain distinct colon-bearing constructs.
 
+### SQL reference persistence and statement carry
+
+SQL `@@` references use the registered `system_variable` kind in both writer
+validation and SQLite CHECK constraints. Opening an existing database for indexing
+upgrades older kind constraints while preserving its reference rows. After an older
+writer fails on these references, rerun normal `cdidx index <projectPath>` with the
+fixed binary; deleting the database or forcing `--rebuild` is unnecessary.
+
+SQL reference extraction clears statement carry at standalone `GO` batch separators
+(including positive repeat counts and trailing comments). It also clears a preceding
+INSERT, UPDATE, DELETE, or SET prefix before an unambiguous independent data statement
+when parentheses are balanced. This prevents repeated seed statements without
+semicolons from rescanning all preceding statements. Compound WITH/MERGE/CREATE
+prefixes, nested expressions, and upsert continuation clauses retain their context;
+temporary-object names survive batch boundaries.
+
 ### GPU and shader reference extraction
 
 CUDA, GLSL, HLSL, Metal, and WGSL use a request-scoped, stateless reference
@@ -6941,6 +6957,19 @@ TypeScript decorator は decorator 名を `annotation` 行として出力し、d
 
 C# の `overwrite:` のような named-argument label は構文であり、型位置ではないため `type_reference` 行を出力してはならない。declaration-type scanner は argument fragment の先頭にある単一 colon の label を、comma で終わる複数行 argument も含めて読み飛ばし、argument value 内の式参照、named `out` declaration の型、明示型 lambda / anonymous method の parameter、および型付き LINQ range variable を維持する。複数行 property subpattern でも property label 後の型を維持する。alias-qualified name（`Alias::Type`）、statement / `case` label、nullable type、ternary expression は別の colon 構文として扱う。
 
+### SQL 参照の保存と文の持ち越し
+
+SQL の `@@` 参照は、writer の検証と SQLite CHECK 制約の両方に登録した
+`system_variable` 種別で保存します。既存DBをインデックス用に開く際、古い種別制約は
+参照行を保持して更新します。旧版がこの参照の保存に失敗した場合は、修正版で通常の
+`cdidx index <projectPath>` を再実行してください。DBの削除や `--rebuild` は不要です。
+
+SQL 参照抽出は、独立した `GO` バッチ区切り（正の繰り返し回数、末尾コメントも含む）
+で文の持ち越しを解消します。先行文が INSERT／UPDATE／DELETE／SET で括弧が閉じている
+場合も、明確に独立したデータ操作文の前で解消します。これにより、セミコロンのない
+初期データ文が過去の全文を繰り返し解析するのを防ぎます。WITH／MERGE／CREATE、
+入れ子の式、upsert の継続句は文脈を維持し、一時オブジェクト名もバッチをまたいで保持します。
+
 ### GPU / shader の参照抽出
 
 CUDA、GLSL、HLSL、Metal、WGSL は request ごとの stateless な参照 extractor を使う。

TESTING_GUIDE.md

@@ -1,5 +1,16 @@
 # Testing Guide
 
+SQL indexing regressions in `IndexCommandRunnerSqlTests`, `DbSchemaConstraintTests`
+and `ReferenceExtractorTests.Extract_SQL_SystemVariablesEmitsSystemVariableReferences`
+cover `@@` kind validation, fresh/full/scoped persistence, coordinates and existing
+CHECK-constraint migration without losing reference rows. `SqlReferenceStatementBoundaryTests`
+checks bounded carry and exact target counts for repeated DML with semicolons, GO,
+or neither; retain multiline, CTE, MERGE, upsert, literal and temp-object controls.
+Run these with SQL symbol/graph and raw/fresh reference persistence tests on net8/net9.
+SQL の `@@` 種別・新規／全体／限定更新・座標・既存DBの制約更新を両runtimeで検証します。
+繰り返しDMLの性能回帰は時間の閾値ではなく持ち越し量と参照件数で検出し、複数行SQLの
+文脈と一時オブジェクト、文字列内の GO を維持してください。
+
 Run `python3 -m unittest discover -s .agent_harness/tests` for shared command-guard
 changes. The existing policy-contract matrix checks the core, Codex PreToolUse and
 PermissionRequest, and Claude PreToolUse without executing the tested commands.

changelog.d/unreleased/+sql-reference-indexing.fixed.md

@@ -0,0 +1,17 @@
+---
+category: fixed
+affected:
+  - src/CodeIndex/Models/SymbolKindCatalog.cs
+  - src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.cs
+  - src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.StatementBoundaries.cs
+---
+
+## English
+
+- Fixed SQL indexing failures when saving `@@ROWCOUNT`, `@@IDENTITY`, and other `system_variable` references. Existing database kind constraints upgrade automatically on the next index run while preserving stored rows.
+- Avoid repeatedly rescanning preceding seed statements at `GO` boundaries and between independent INSERT/UPDATE/DELETE statements without semicolons. Multiline SQL and compound-statement context remain intact.
+
+## 日本語
+
+- `@@ROWCOUNT`、`@@IDENTITY` などの `system_variable` 参照を保存する際にSQLインデックスが失敗する問題を修正しました。既存DBの種別制約は次回のインデックス実行時に保存済みの行を維持して自動更新します。
+- `GO` 区切りやセミコロンのない独立した INSERT／UPDATE／DELETE 文で、過去の初期データ文を繰り返し再解析する問題を修正しました。複数行SQLと複合文の文脈は維持します。

src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.StatementBoundaries.cs

@@ -0,0 +1,84 @@
+using System.Globalization;
+using System.Text.RegularExpressions;
+using Regex = CodeIndex.Indexer.BoundedRegex;
+
+namespace CodeIndex.Indexer;
+
+internal static partial class SqlReferenceExtractor
+{
+    // Only unambiguous data-statement starts qualify: UPDATE SET (MERGE / ON CONFLICT)
+    // and UPDATE column = value (ON DUPLICATE KEY) are continuations, not boundaries.
+    // MERGE / ON CONFLICT の UPDATE SET や ON DUPLICATE KEY の UPDATE column = value は
+    // 継続句なので、独立した DML 文の開始だけを境界にする。
+    private static readonly Regex IndependentDataStatementStartRegex = new(
+        $@"^\s*(?:UPDATE\s+{QualifiedIdentifierNoCapturePattern}\s+SET\b|INSERT\s+(?:INTO\s+)?{QualifiedIdentifierNoCapturePattern}|DELETE\s+FROM\s+{QualifiedIdentifierNoCapturePattern})",
+        RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
+
+    private static bool IsBatchSeparator(string line)
+    {
+        var text = line.AsSpan().Trim();
+        if (text.Length < 2 || !text[..2].Equals("GO", StringComparison.OrdinalIgnoreCase))
+            return false;
+        if (text.Length == 2)
+            return true;
+        return char.IsWhiteSpace(text[2])
+            && int.TryParse(text[2..].Trim(), NumberStyles.None, CultureInfo.InvariantCulture, out var count)
+            && count > 0;
+    }
+
+    private static bool ShouldFlushDataStatementPrefixAtLineBoundary(string prefix, string nextLine)
+    {
+        var start = SkipWhitespaceAhead(prefix, 0);
+        // Keep compound statements (WITH, MERGE, CREATE, etc.) intact. Their later
+        // lines can depend on earlier clauses even outside parentheses.
+        // WITH / MERGE / CREATE などは前の句が後続行の解析に必要なので維持する。
+        if (!IsKeywordAt(prefix, start, "INSERT")
+            && !IsKeywordAt(prefix, start, "UPDATE")
+            && !IsKeywordAt(prefix, start, "DELETE")
+            && !IsKeywordAt(prefix, start, "SET"))
+        {
+            return false;
+        }
+
+        if (!IndependentDataStatementStartRegex.IsMatch(nextLine))
+            return false;
+
+        // The input is already comment/string masked. Quoted identifiers must still
+        // be skipped so parentheses in names cannot create a false statement boundary.
+        // 入力の comment / string はマスク済み。引用識別子内の括弧も境界に数えない。
+        var depth = 0;
+        for (var index = start; index < prefix.Length; index++)
+        {
+            var value = prefix[index];
+            if (value is '[' or '"' or '`')
+            {
+                var closing = value == '[' ? ']' : value;
+                var closed = false;
+                while (++index < prefix.Length)
+                {
+                    if (prefix[index] != closing)
+                        continue;
+                    if (index + 1 < prefix.Length && prefix[index + 1] == closing)
+                    {
+                        index++;
+                        continue;
+                    }
+                    closed = true;
+                    break;
+                }
+                if (!closed)
+                    return false;
+            }
+            else if (value == '(')
+            {
+                depth++;
+            }
+            else if (value == ')' && --depth < 0)
+            {
+                return false;
+            }
+        }
+
+        return depth == 0;
+    }
+}

src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.cs

@@ -142,11 +142,16 @@ public static HashSet<int> Emit(
         if (string.IsNullOrWhiteSpace(lineFragment))
             return suppressedCallIndices;
 
-        if (ShouldFlushTempObjectPrefixAtLineBoundary(state.StatementPrefix, lineFragment))
+        var isBatchSeparator = IsBatchSeparator(lineFragment);
+        if (isBatchSeparator
+            || ShouldFlushDataStatementPrefixAtLineBoundary(state.StatementPrefix, lineFragment)
+            || ShouldFlushTempObjectPrefixAtLineBoundary(state.StatementPrefix, lineFragment))
         {
             CollectTempObjectNamesFromStatement(state.StatementPrefix, state.EstablishedTempObjectNames);
             state.StatementPrefix = string.Empty;
         }
+        if (isBatchSeparator)
+            return suppressedCallIndices;
 
         var combinedLine = CombineStatementPrefix(state.StatementPrefix, lineFragment, out var lineOffset);
         int statementStart = 0;

src/CodeIndex/Models/SymbolKindCatalog.cs

@@ -138,6 +138,7 @@ public static class SymbolKindCatalog
         "stage",
         "razor_event_binding",
         "subscribe",
+        "system_variable",
         "type_reference",
         "type_tag",
         "unsubscribe",

tests/CodeIndex.Tests/DbSchemaConstraintTests.cs

@@ -93,6 +93,59 @@ public void InitializeSchema_KindCheckConstraintsMatchCatalog_Issue4178()
         }
     }
 
+    [Fact]
+    public void InitializeSchema_AddsSqlSystemVariableKindToExistingReferenceChecks()
+    {
+        var dbDir = TestProjectHelper.CreateTempProject("codeindex_sql_kind_migration");
+        var dbPath = Path.Combine(dbDir, "codeindex.db");
+        try
+        {
+            using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath))
+                db.InitializeSchema();
+
+            using (var conn = new SqliteConnection(new SqliteConnectionStringBuilder
+            { DataSource = dbPath, Pooling = false }.ConnectionString))
+            {
+                conn.Open();
+                var legacySql = ReadCreateSql(conn, "symbol_references")
+                    .Replace("'system_variable', ", "", StringComparison.Ordinal);
+                Exec(conn, "PRAGMA foreign_keys=OFF; DROP TABLE symbol_references;");
+                Exec(conn, legacySql);
+                Exec(conn, """
+                    PRAGMA foreign_keys=ON;
+                    INSERT INTO files (id, path, lang) VALUES (1, 'SampleData.sql', 'sql');
+                    INSERT INTO reference_lines (id, file_id, line, context) VALUES (9, 1, 1, 'EXEC SaveOrder;');
+                    INSERT INTO symbol_references (id, file_id, symbol_name, reference_kind, line, reference_line_id)
+                    VALUES (7, 1, 'SaveOrder', 'call', 1, 9);
+                    """);
+                Assert.Throws<SqliteException>(() => Exec(conn, """
+                    INSERT INTO symbol_references (file_id, symbol_name, reference_kind, line)
+                    VALUES (1, '@@ROWCOUNT', 'system_variable', 2)
+                    """));
+            }
+
+            using var migrated = new DbContext(DbOpenIntent.WriteIndex, dbPath);
+            migrated.InitializeSchema();
+            var writer = new DbWriter(migrated.Connection);
+            writer.InsertReferences([
+                new ReferenceRecord { FileId = 1, SymbolName = "@@ROWCOUNT", ReferenceKind = "system_variable", Line = 2 }
+            ]);
+            Assert.Equal(1L, CountRows(migrated.Connection, "symbol_references", "id = 7 AND reference_line_id = 9 AND reference_kind = 'call'"));
+            Assert.Equal(1L, CountRows(migrated.Connection, "symbol_references", "symbol_name = '@@ROWCOUNT' AND reference_kind = 'system_variable'"));
+            AssertSameSet(SymbolKindCatalog.ReferenceKinds,
+                ExtractCheckValues(ReadCreateSql(migrated.Connection, "symbol_references"), "reference_kind"));
+            using var check = migrated.Connection.CreateCommand();
+            check.CommandText = "PRAGMA foreign_key_check";
+            Assert.Null(check.ExecuteScalar());
+            check.CommandText = "PRAGMA integrity_check";
+            Assert.Equal("ok", check.ExecuteScalar());
+        }
+        finally
+        {
+            TestProjectHelper.DeleteDirectory(dbDir);
+        }
+    }
+
     [Fact]
     public void InitializeSchema_PublicTaxonomyMutationCannotSplitCanonicalContracts()
     {

tests/CodeIndex.Tests/IndexCommandRunnerSqlTests.cs

@@ -0,0 +1,61 @@
+using CodeIndex.Cli;
+using CodeIndex.Database;
+
+namespace CodeIndex.Tests;
+
+public partial class IndexCommandRunnerTests
+{
+    [Fact]
+    public void Run_SqlSystemVariablesPersistAcrossFreshFullAndScopedUpdates()
+    {
+        var projectRoot = CreateTempProject();
+        var dbPath = Path.Combine(projectRoot, ".cdidx", "codeindex.db");
+        const string content = """
+            CREATE PROCEDURE dbo.SaveOrder
+            AS
+            BEGIN
+                SELECT @@IDENTITY;
+                IF @@ROWCOUNT = 0 SELECT @@ERROR;
+                SELECT @@session.sql_mode, @@global.max_connections;
+            END;
+            """;
+        try
+        {
+            var path = Path.Combine(projectRoot, "SampleData.sql");
+            for (var pass = 0; pass < 3; pass++)
+            {
+                File.WriteAllText(path, content + new string('\n', pass));
+                string[] arguments = pass == 2
+                    ? [projectRoot, "--files", "SampleData.sql", "--db", dbPath, "--json", "--quiet"]
+                    : [projectRoot, "--db", dbPath, "--json", "--quiet"];
+                var (exitCode, json) = RunAndCaptureJson(arguments);
+                Assert.Equal(CommandExitCodes.Success, exitCode);
+                Assert.Equal("success", json.GetProperty("status").GetString());
+                Assert.True(json.GetProperty("index_complete").GetBoolean());
+                Assert.True(json.GetProperty("reference_graph_complete").GetBoolean());
+
+                using var connection = OpenNonPoolingConnection(dbPath);
+                connection.Open();
+                using var command = connection.CreateCommand();
+                command.CommandText = """
+                    SELECT symbol_name, line, column_number
+                    FROM symbol_references
+                    WHERE reference_kind = 'system_variable'
+                    ORDER BY line, column_number
+                    """;
+                using var reader = command.ExecuteReader();
+                var actual = new List<(string Name, int Line, int Column)>();
+                while (reader.Read())
+                    actual.Add((reader.GetString(0), reader.GetInt32(1), reader.GetInt32(2)));
+                Assert.Equal(
+                    [("@@IDENTITY", 4, 12), ("@@ROWCOUNT", 5, 8), ("@@ERROR", 5, 30),
+                     ("@@session.sql_mode", 6, 12), ("@@global.max_connections", 6, 32)],
+                    actual);
+            }
+        }
+        finally
+        {
+            DeleteDirectory(projectRoot);
+        }
+    }
+}

tests/CodeIndex.Tests/ReferenceExtractorTests.cs

@@ -9741,6 +9741,7 @@ CREATE PROCEDURE dbo.SaveOrder
         Assert.Contains(references, r => r.SymbolName == "@@ERROR" && r.ReferenceKind == "system_variable");
         Assert.Contains(references, r => r.SymbolName == "@@session.sql_mode" && r.ReferenceKind == "system_variable");
         Assert.Contains(references, r => r.SymbolName == "@@global.max_connections" && r.ReferenceKind == "system_variable");
+        Assert.All(references, r => Assert.True(SymbolKindCatalog.IsValidReferenceKind(r.ReferenceKind), r.ReferenceKind));
     }
 
     [Fact]