[Perf] Bypass SqlBulkCopy Graph Column Mapping if Not Copying Graph Tables

[benrr101]

Description

This addresses a performance regression in the DataTypeReader / Async perf suites, indirectly.

In #3677, SQL Graph column alias mapping support was added. While this works great for SQL Graph tables, it was a bit over eager and invoked the entire graph table mapping behavior even when the tables involved did not contain SQL Graph tables. Thus, every bulk copy implicitly generated mapping tables, etc. This dramatically decreased perf in scenarios where simple tables were being bulk copied.

To resolve this, we introduce a mechanism to bypass the graph column alias table generation when the table does not contain any of the graph alias columns ($edge_id, $to_id, $from_id, $node_id). This returns performance in the DataTypeReader/Async perf suites to pre-#3677 levels.

It is worth nothing that this was discovered as an artifact of the way the DataTypeReader/Async perf suites are implemented. Although the suite aims to verify perf of reading various data types from a SqlDataReader, majority of the time spent running the test is spent doing setup and teardown steps - ie, SqlBulkCopy of data into the test table. So although this change doesn't actually impact the SqlDataReader performance, it does resolve the perf suite regression, and improve perf of SqlBulkCopy in the most common pathways.

🤖

• Identification of problematic commits between 7.0 and current main
• Test of reverting Feature | Support SQL Graph column aliases in SqlBulkCopy #3677, which demonstrated resolution of regression
• Implementation of mechanism to bypass graph alias table construction when bulk copying simple tables
• Comparison of perf from before and after changes

Issues

N/A

Testing

Local comparison of perf suite before and after these changes suggest a significant improvement.

[Copilot]

Pull request overview

This PR improves SqlBulkCopy performance by avoiding SQL Graph column-alias resolution work (temp table creation + sys catalog scans + extra resultset parsing) when the bulk copy operation isn’t using SQL Graph alias destination columns ($edge_id, $to_id, $from_id, $node_id). This targets a regression observed in perf suites where bulk-copy setup/teardown dominated runtime.

Changes:

• Conditionally generate/execute the #Column_Aliases temp table + population queries only when destination column mappings reference SQL Graph alias names.
• Make cached-metadata reuse aware of whether the column-alias resultset is required/present, and guard alias-application logic on resultset presence.
• Update ManualTests statistics expectations to reflect the reduced query/DDL/DML work in the common non-graph pathway.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
src/Microsoft.Data.SqlClient/tests/ManualTests/BulkCopy/CopyAllFromReader.cs	Updates expected SqlConnection statistics counters to match the optimized non-graph bulk copy behavior.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs	Bypasses graph alias temp-table/query generation unless needed; adds result-count checks for cached metadata and alias resultset handling.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs:156

• The comment implies the initial query may return the column-alias result set based on destination table shape, but the new behavior is actually conditional on whether column mappings reference SQL Graph alias names (see ShouldResolveColumnAliases()). Clarifying this prevents confusion when reading stats/result-set expectations.

        // The initial query will return three tables, and may return a fourth for column aliases.

[cheenamalhotra]

I think test failures are related to changes here, please take a look.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[codecov]

Codecov Report

❌ Patch coverage is 92.85714% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.99%. Comparing base (ee529d4) to head (fc7fdd5).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...Client/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs	92.85%	8 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4535      +/-   ##
==========================================
- Coverage   64.78%   62.99%   -1.79%     
==========================================
  Files         288      283       -5     
  Lines       44418    67467   +23049     
==========================================
+ Hits        28774    42499   +13725     
- Misses      15644    24968    +9324     

Flag	Coverage Δ
CI-SqlClient	?
PR-SqlClient-Project	62.99% <92.85%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[edwardneal]

A few comments - one nit and one slightly more substantial.

Do you have a before/after benchmark? I agree that making a change to the SQL statement would improve performance, but if something isn't a Graph table then the result set should have zero rows and have no direct performance impact.

[benrr101]

@edwardneal we do have a before/after benchmark and this does make a marked improvement (ie, restores performance to pre-graph column mapping levels). Unfortunately, it's an internal benchmark dashboard. But, trust me, it's way better 😃

[benrr101]

@priyankatiwari08 feedback has been addressed

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs:530

• When alias resolution is not needed (_localColumnMappingsResolveAliases == false), the initial query still creates #Column_Aliases and always returns the column-alias result set (empty). This keeps an extra result set/SELECT in the metadata round-trip, which can still show up in connection statistics and reduces the benefit of bypassing graph-alias work. Consider omitting both the temp-table creation and the final SELECT/DROP when alias resolution is not required; AnalyzeTargetAndCreateUpdateBulkCommand already guards on internalResults.Count.

            string selectColumnAliasesQuery = """

SELECT [Canonical_Column_Name], [Aliased_Column_Name]
FROM #Column_Aliases
ORDER BY [Canonical_Column_Id] ASC

DROP TABLE #Column_Aliases
""";

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs:2401

• WriteRowSourceToServerCommon adds an extra foreach over SqlBulkCopyColumnMappingCollection just to reset MappedDestinationColumn. Since SqlBulkCopyColumnMappingCollection derives from CollectionBase, this enumerates via a non-generic IEnumerator and can allocate; it also doubles the traversal immediately before another foreach. This can be folded into a single indexed loop to avoid the extra enumeration and keep the per-operation overhead lower.

                foreach (SqlBulkCopyColumnMapping bulkCopyColumn in _localColumnMappings)
                {
                    bulkCopyColumn.MappedDestinationColumn = null;
                }

[mdaigle]

Can you link the sqlclient-perf-pr run that shows the improvements against main?

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

[mdaigle]

Looks good. Would still like to see the perf run numbers :)