Skip to content

[fix](iceberg) Fix execute action validation gaps#61381

Open
suxiaogang223 wants to merge 2 commits intoapache:masterfrom
suxiaogang223:codex/fix-iceberg-execute-action-validation
Open

[fix](iceberg) Fix execute action validation gaps#61381
suxiaogang223 wants to merge 2 commits intoapache:masterfrom
suxiaogang223:codex/fix-iceberg-execute-action-validation

Conversation

@suxiaogang223
Copy link
Contributor

@suxiaogang223 suxiaogang223 commented Mar 16, 2026

What problem does this PR solve?

Issue Number: N/A

Related PR: N/A

Problem Summary:

  • Fix rollback_to_timestamp so epoch millis input is parsed correctly instead of falling through to rollbackToTime(-1).
  • Reject invalid rewrite_data_files input when min-file-size-bytes > max-file-size-bytes during FE validation.
  • Add Iceberg regression coverage for the epoch-millis rollback path and invalid rewrite_data_files file-size bounds.

Release note

None

Check List (For Author)

  • Test
    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

Regression test:

  • bash run-regression-test.sh --run -d external_table_p0/iceberg/action -s test_iceberg_optimize_actions_ddl

  • In the current local environment, the regression framework ran successfully but the Iceberg suite was skipped because enableIcebergTest is disabled, so the new Iceberg cases were not executed locally.

  • Behavior changed:

    • No.
    • Yes. Buggy Iceberg execute action inputs are now rejected or interpreted correctly instead of silently succeeding or executing with -1.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Mar 16, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 26714 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e64c68badb7065d04157f177b604dd368b0bd707, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17657	4511	4300	4300
q2	q3	10644	760	523	523
q4	4679	360	250	250
q5	7560	1196	1035	1035
q6	197	176	146	146
q7	785	831	661	661
q8	9343	1472	1300	1300
q9	4926	4767	4704	4704
q10	6329	1916	1649	1649
q11	456	270	234	234
q12	726	575	467	467
q13	18036	2968	2161	2161
q14	224	236	215	215
q15	q16	739	738	660	660
q17	720	820	441	441
q18	5867	5363	5259	5259
q19	1142	980	612	612
q20	543	487	387	387
q21	4524	1814	1402	1402
q22	360	535	308	308
Total cold run time: 95457 ms
Total hot run time: 26714 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4703	4702	4570	4570
q2	q3	3888	4448	3893	3893
q4	936	1200	804	804
q5	4112	4434	4361	4361
q6	181	167	138	138
q7	1755	1633	1537	1537
q8	2437	2688	2540	2540
q9	7720	7431	7620	7431
q10	3753	3999	3721	3721
q11	498	436	424	424
q12	478	603	456	456
q13	2714	3282	2623	2623
q14	285	307	283	283
q15	q16	748	809	737	737
q17	1153	1367	1341	1341
q18	7308	6779	6567	6567
q19	889	841	908	841
q20	2089	2196	1973	1973
q21	3948	3463	3243	3243
q22	472	423	444	423
Total cold run time: 50067 ms
Total hot run time: 47906 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 168533 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e64c68badb7065d04157f177b604dd368b0bd707, data reload: false

query5	4331	654	499	499
query6	324	223	206	206
query7	4226	473	262	262
query8	339	246	231	231
query9	8714	2747	2738	2738
query10	537	412	334	334
query11	7014	5091	4871	4871
query12	184	128	126	126
query13	1265	460	362	362
query14	5719	3722	3453	3453
query14_1	2872	2852	2775	2775
query15	201	192	175	175
query16	977	476	457	457
query17	908	727	624	624
query18	2459	453	355	355
query19	212	211	186	186
query20	138	128	129	128
query21	219	132	112	112
query22	13266	14179	14642	14179
query23	16156	15880	15942	15880
query23_1	15713	15711	15750	15711
query24	7514	1652	1248	1248
query24_1	1217	1215	1216	1215
query25	572	555	416	416
query26	1237	264	155	155
query27	2831	475	295	295
query28	4485	1851	1849	1849
query29	810	547	478	478
query30	301	231	189	189
query31	1025	945	872	872
query32	78	69	68	68
query33	504	332	291	291
query34	895	870	519	519
query35	632	678	597	597
query36	1095	1098	948	948
query37	127	94	80	80
query38	2968	2954	2877	2877
query39	851	832	821	821
query39_1	804	805	791	791
query40	229	155	139	139
query41	63	100	59	59
query42	263	255	257	255
query43	236	251	220	220
query44	
query45	194	190	180	180
query46	869	978	595	595
query47	2148	2154	2080	2080
query48	306	307	233	233
query49	630	451	379	379
query50	678	270	212	212
query51	4131	4056	4086	4056
query52	258	266	255	255
query53	289	340	288	288
query54	299	276	261	261
query55	90	88	86	86
query56	306	312	304	304
query57	1945	1823	1679	1679
query58	284	269	267	267
query59	2800	2920	2747	2747
query60	345	326	315	315
query61	155	155	140	140
query62	630	582	539	539
query63	307	280	279	279
query64	5013	1262	965	965
query65	
query66	1454	455	344	344
query67	24179	24528	24274	24274
query68	
query69	397	324	278	278
query70	979	972	992	972
query71	342	302	297	297
query72	2746	2704	2152	2152
query73	547	546	310	310
query74	9617	9662	9425	9425
query75	2839	2726	2495	2495
query76	2277	1043	690	690
query77	352	376	315	315
query78	10908	11162	10467	10467
query79	1115	851	571	571
query80	1342	615	528	528
query81	543	261	227	227
query82	1112	148	126	126
query83	337	260	241	241
query84	293	120	92	92
query85	894	477	434	434
query86	422	350	321	321
query87	3187	3083	2985	2985
query88	3520	2646	2657	2646
query89	424	379	356	356
query90	2029	178	174	174
query91	170	158	137	137
query92	77	76	70	70
query93	918	859	491	491
query94	639	330	296	296
query95	586	332	318	318
query96	634	515	231	231
query97	2512	2525	2409	2409
query98	239	221	217	217
query99	1028	988	929	929
Total cold run time: 248142 ms
Total hot run time: 168533 ms

@suxiaogang223
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 27020 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 30531d85a0f39f8afc4fed43975dc46918e64580, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17582	4486	4303	4303
q2	q3	10640	809	533	533
q4	4684	361	255	255
q5	7573	1219	1034	1034
q6	175	174	148	148
q7	798	843	683	683
q8	9288	1487	1359	1359
q9	4907	4760	4708	4708
q10	6256	1942	1702	1702
q11	477	257	257	257
q12	732	589	475	475
q13	18053	2923	2185	2185
q14	236	231	207	207
q15	q16	727	733	672	672
q17	745	845	455	455
q18	6007	5295	5308	5295
q19	1111	991	630	630
q20	556	517	385	385
q21	4471	1845	1437	1437
q22	566	372	297	297
Total cold run time: 95584 ms
Total hot run time: 27020 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4760	4641	4604	4604
q2	q3	3940	4348	3852	3852
q4	919	1386	757	757
q5	4102	4434	4342	4342
q6	200	176	143	143
q7	1815	1663	1534	1534
q8	2513	2790	2563	2563
q9	7738	7353	7576	7353
q10	3791	4042	3605	3605
q11	516	454	426	426
q12	504	609	462	462
q13	2697	3163	2400	2400
q14	298	304	276	276
q15	q16	747	772	719	719
q17	1178	1319	1343	1319
q18	7193	6776	6856	6776
q19	941	942	939	939
q20	2076	2146	2027	2027
q21	3978	3539	3523	3523
q22	458	445	388	388
Total cold run time: 50364 ms
Total hot run time: 48008 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 168513 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 30531d85a0f39f8afc4fed43975dc46918e64580, data reload: false

query5	4325	645	496	496
query6	328	230	207	207
query7	4218	468	263	263
query8	342	245	228	228
query9	8709	2723	2736	2723
query10	525	418	347	347
query11	7023	5118	4872	4872
query12	185	126	127	126
query13	1286	468	350	350
query14	5765	3855	3505	3505
query14_1	2922	2875	2852	2852
query15	213	193	178	178
query16	977	478	454	454
query17	903	737	633	633
query18	2467	469	355	355
query19	221	210	190	190
query20	142	133	128	128
query21	220	141	113	113
query22	13326	14065	14670	14065
query23	16375	15818	15633	15633
query23_1	15881	15425	15356	15356
query24	7221	1630	1243	1243
query24_1	1247	1236	1238	1236
query25	576	463	411	411
query26	1246	261	154	154
query27	2768	485	300	300
query28	4474	1866	1874	1866
query29	884	577	484	484
query30	300	228	191	191
query31	1006	940	870	870
query32	79	73	69	69
query33	521	341	287	287
query34	907	912	525	525
query35	664	684	622	622
query36	1073	1140	976	976
query37	139	95	81	81
query38	2967	2899	2859	2859
query39	864	841	812	812
query39_1	801	814	810	810
query40	232	153	136	136
query41	64	62	61	61
query42	264	256	260	256
query43	241	242	221	221
query44	
query45	198	192	184	184
query46	913	1008	618	618
query47	3341	2158	2076	2076
query48	318	322	236	236
query49	630	469	374	374
query50	717	290	211	211
query51	4076	4083	4010	4010
query52	266	269	261	261
query53	301	348	286	286
query54	302	280	268	268
query55	90	91	83	83
query56	327	337	332	332
query57	1901	1784	1718	1718
query58	288	282	296	282
query59	2816	2965	2742	2742
query60	350	343	331	331
query61	157	157	159	157
query62	621	584	543	543
query63	315	284	279	279
query64	5167	1305	1038	1038
query65	
query66	1465	473	380	380
query67	24379	24380	24196	24196
query68	
query69	405	325	289	289
query70	994	951	972	951
query71	338	309	341	309
query72	2874	2710	2422	2422
query73	548	556	321	321
query74	9685	9583	9364	9364
query75	2884	2757	2470	2470
query76	2296	1055	696	696
query77	368	380	323	323
query78	10994	11050	10469	10469
query79	1241	816	573	573
query80	1367	627	539	539
query81	558	275	229	229
query82	1034	159	124	124
query83	379	270	247	247
query84	301	124	103	103
query85	917	533	474	474
query86	445	315	295	295
query87	3167	3149	3021	3021
query88	3689	2685	2676	2676
query89	429	371	348	348
query90	1967	188	168	168
query91	176	163	141	141
query92	77	80	71	71
query93	1043	919	504	504
query94	655	318	289	289
query95	596	418	318	318
query96	663	545	231	231
query97	2485	2489	2415	2415
query98	246	228	224	224
query99	1035	995	914	914
Total cold run time: 251611 ms
Total hot run time: 168513 ms

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 20, 2026
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.x dev/4.1.x reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants