[fix](regression) Stabilize rowsets timestamp filters#65050
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
/review |
There was a problem hiding this comment.
Review result: requesting changes for one test-stability gap.
Critical checkpoint conclusions:
- Goal/test proof: the changed test now captures a stable lower bound before table creation and keeps the expected rowset versions unchanged.
- Scope: the edit is small, but the same timestamp-boundary pattern remains in a parallel rowsets regression.
- Concurrency/lifecycle/config/compatibility/persistence: no product-code, config, protocol, lifecycle, or storage-format changes in this PR.
- Parallel paths: M-1 identifies the remaining cloud-only rowsets scan test path with the same second-level
>= nowrace. - Test results: PR body reports query_p0/system test_query_sys_rowsets runs including 20 consecutive runs; I did not run local regression here because this checkout has no
thirdparty/installed. - User focus: no additional focus was supplied.
Subagent conclusions: optimizer-rewrite and tests-session-config both reported no candidates in their initial passes. After M-1 was added to the ledger/proposed comment set, convergence round 1 ended with both live subagents replying NO_NEW_VALUABLE_FINDINGS.
### What problem does this PR solve?
Issue Number: None
Problem Summary: The system rowsets regressions used second-level current timestamps as lower bounds for NEWEST_WRITE_TIMESTAMP. In one test, the timestamp was captured after the table was created and could filter out the initial rowset on a boundary race. In the cloud-only scan variant, the timestamp could be equal to the create-table rowset timestamp and unexpectedly include version 0-1. Capture stable lower bounds on the intended side of table creation for each case, so the tests still exercise timestamp filters while avoiding second-level boundary races.
### Release note
None
### Check List (For Author)
- Test:
- Regression test: doris-local-regression --network 10.26.20.3/24 all -d query_p0/system -s test_query_sys_rowsets
- Regression test: doris-local-regression --network 10.26.20.3/24 run -d query_p0/system -s test_query_sys_rowsets (200 consecutive runs)
- Regression test: doris-local-regression --network 10.26.20.3/24 run -d query_p0/system -s test_query_sys_scan_rowsets (local non-cloud guard path)
- Regression test: doris-local-regression --network 10.26.20.3/24 run -d query_p0/system -s test_query_sys_rowsets
- Behavior changed: No
- Does this need documentation: No
|
/review |
cceab75 to
f4c3543
Compare
There was a problem hiding this comment.
Review completed for PR 65050 at f4c3543.
I did not find any new blocking issue or any new valuable inline comment beyond the existing thread that has already been addressed by the current patch.
Critical checkpoint conclusions:
- Goal and proof: the PR targets second-level
NEWEST_WRITE_TIMESTAMPboundary flakiness in the rowsets regressions. The normal rowsets test now captures a lower bound before table creation, preserving the expectedrowsets4output that includes0-1through5-5; the cloud scan variant now waits past the create-table rowset second before capturing the lower bound, preserving the expectedrowsets4output that excludes0-1. - Scope and focus: the live GitHub PR file list and compare API show only the two rowsets regression suites. The change is small and focused on timestamp filter stabilization.
- Parallel paths: the sibling cloud-only scan test is now updated, so the existing inline concern about leaving
test_query_sys_scan_rowsets.groovyflaky is covered by the current head. - Tests and expected results: the expected-output files already encode the intended difference between the local and cloud scan variants. I verified the changed queries against those expectations and the BE rowsets scanner path that exposes
newest_write_timestampas a DATETIME from Unix-second rowset metadata. - Concurrency, lifecycle, configs, compatibility, transactions, and observability: this is regression-test-only code and does not add runtime product state, config, persistence, protocol, or logging behavior.
- Performance: the added 1.1s sleep is limited to the cloud-only regression path and is directly tied to the second-level timestamp boundary being tested.
- Hygiene:
git diff --checkis clean for the true PR parent-to-head range over the changed files.
User focus: no additional user-provided review focus was present.
Subagent conclusions:
optimizer-rewritereportedNO_NEW_VALUABLE_FINDINGS; no optimizer/rewrite candidate became an inline comment.tests-session-configreportedNO_NEW_VALUABLE_FINDINGS; no test/config candidate became an inline comment.- Main merged
MAIN-DUP-001as a duplicate/resolution note for the existing inline thread about the cloud-only sibling test. - Convergence round 1 ended with both live subagents reporting
NO_NEW_VALUABLE_FINDINGSfor the same empty proposed final comment set after the last ledger update.
|
run buildall |
|
PR approved by at least one committer and no changes requested. |
What problem does this PR solve?
Issue Number: None
Problem Summary: The system rowsets regressions used second-level current timestamps as lower bounds for
NEWEST_WRITE_TIMESTAMP. In one test, the timestamp was captured after the table was created and could filter out the initial rowset on a boundary race. In the cloud-only scan variant, the timestamp could be equal to the create-table rowset timestamp and unexpectedly include version0-1. Capture stable lower bounds on the intended side of table creation for each case, so the tests still exercise timestamp filters while avoiding second-level boundary races.Release note
None
Check List (For Author)
./run-regression-test.sh --run -d query_p0/system -s test_query_sys_rowsets./run-regression-test.sh --run -d query_p0/system -s test_query_sys_rowsets(200 consecutive runs)./run-regression-test.sh --run -d query_p0/system -s test_query_sys_scan_rowsets(local non-cloud guard path)./run-regression-test.sh --run -d query_p0/system -s test_query_sys_rowsets