Commit 75fdec3
### What changes were proposed in this pull request?
Add `numSourceRows` metric for `MergeIntoExec`, from source node's `numOutputRows`.
Assumption is that all child nodes have `numOutputRows`. If not found, `numSourceRows` would be -1.
### Why are the changes needed?
Improve completeness and debuggability of Merge Into metrics.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Unit test cases for numSourceNodes metric.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #52669 from asl3/merge-numsourcerows.
Authored-by: Amanda Liu <amanda.liu@databricks.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent fb8dbd7 commit 75fdec3
File tree
4 files changed
+346
-227
lines changed- sql
- catalyst/src/main
- java/org/apache/spark/sql/connector/write
- scala/org/apache/spark/sql/connector/write
- core/src
- main/scala/org/apache/spark/sql/execution/datasources/v2
- test/scala/org/apache/spark/sql/connector
4 files changed
+346
-227
lines changedLines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
30 | 35 | | |
31 | 36 | | |
32 | 37 | | |
| |||
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| |||
Lines changed: 38 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
| 34 | + | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| 38 | + | |
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
| |||
481 | 482 | | |
482 | 483 | | |
483 | 484 | | |
| 485 | + | |
484 | 486 | | |
| 487 | + | |
485 | 488 | | |
486 | 489 | | |
487 | 490 | | |
| |||
493 | 496 | | |
494 | 497 | | |
495 | 498 | | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
496 | 533 | | |
497 | 534 | | |
498 | 535 | | |
| |||
0 commit comments