Skip to content

[fix] (storage) Fix pointer dereference bug in LocalFileSystem::list_impl#60907

Open
xuchenhao wants to merge 3 commits intoapache:masterfrom
HYDCP:fix-pointer-check
Open

[fix] (storage) Fix pointer dereference bug in LocalFileSystem::list_impl#60907
xuchenhao wants to merge 3 commits intoapache:masterfrom
HYDCP:fix-pointer-check

Conversation

@xuchenhao
Copy link
Contributor

@xuchenhao xuchenhao commented Feb 28, 2026

What problem does this PR solve?

Issue Number: close #60904

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@xuchenhao
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 29110 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d335f7da7036bd1cbabc61ce1028b1ed4e2c0851, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17596	4515	4337	4337
q2	q3	10657	818	558	558
q4	4704	374	246	246
q5	7778	1209	1033	1033
q6	211	173	151	151
q7	801	846	667	667
q8	10553	1505	1343	1343
q9	6095	4787	4709	4709
q10	6864	1897	1634	1634
q11	453	266	242	242
q12	742	564	461	461
q13	17778	4241	3442	3442
q14	235	229	211	211
q15	942	792	799	792
q16	744	721	690	690
q17	749	844	428	428
q18	5985	5314	5245	5245
q19	1201	993	648	648
q20	511	493	555	493
q21	4539	1941	1514	1514
q22	380	308	266	266
Total cold run time: 99518 ms
Total hot run time: 29110 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4859	4652	4641	4641
q2	q3	1861	2281	1814	1814
q4	888	1186	754	754
q5	4041	4501	4407	4407
q6	176	175	146	146
q7	1771	1667	1525	1525
q8	2512	2777	2526	2526
q9	7537	7440	7331	7331
q10	2617	2861	2393	2393
q11	509	429	404	404
q12	577	643	452	452
q13	4111	4486	3587	3587
q14	278	304	287	287
q15	853	825	837	825
q16	709	763	722	722
q17	1207	1566	1312	1312
q18	7296	6894	6628	6628
q19	871	914	945	914
q20	2047	2168	2010	2010
q21	4006	3411	3550	3411
q22	473	451	380	380
Total cold run time: 49199 ms
Total hot run time: 46469 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184082 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d335f7da7036bd1cbabc61ce1028b1ed4e2c0851, data reload: false

query5	4331	638	515	515
query6	338	218	199	199
query7	4202	478	265	265
query8	347	241	248	241
query9	8748	2715	2740	2715
query10	512	380	347	347
query11	17013	17588	17268	17268
query12	205	174	145	145
query13	1296	487	360	360
query14	6715	3288	3043	3043
query14_1	2949	2876	2929	2876
query15	216	195	199	195
query16	4890	491	451	451
query17	1422	732	617	617
query18	2347	434	377	377
query19	220	222	181	181
query20	159	130	122	122
query21	212	143	121	121
query22	5640	5327	5078	5078
query23	17218	16764	16547	16547
query23_1	16719	16576	16688	16576
query24	7374	1614	1204	1204
query24_1	1210	1229	1229	1229
query25	528	465	405	405
query26	1203	261	180	180
query27	2732	492	279	279
query28	4473	1890	1900	1890
query29	739	567	460	460
query30	305	248	212	212
query31	870	717	640	640
query32	82	75	74	74
query33	512	349	278	278
query34	914	925	542	542
query35	627	676	598	598
query36	1083	1152	1020	1020
query37	130	94	92	92
query38	2960	2945	2879	2879
query39	1020	889	851	851
query39_1	828	845	867	845
query40	234	150	140	140
query41	63	69	58	58
query42	110	103	100	100
query43	371	381	356	356
query44	
query45	205	189	184	184
query46	873	980	607	607
query47	2118	2124	2046	2046
query48	304	330	228	228
query49	618	488	377	377
query50	688	314	212	212
query51	4105	4114	4057	4057
query52	105	108	95	95
query53	292	337	282	282
query54	300	274	255	255
query55	87	86	88	86
query56	301	313	300	300
query57	1385	1338	1273	1273
query58	286	276	274	274
query59	2544	2662	2520	2520
query60	345	338	320	320
query61	144	150	145	145
query62	620	581	561	561
query63	318	277	272	272
query64	4419	1237	991	991
query65	
query66	1303	440	354	354
query67	16312	16373	16322	16322
query68	
query69	392	308	273	273
query70	990	976	957	957
query71	326	297	297	297
query72	2820	2946	2532	2532
query73	550	547	323	323
query74	10007	9941	9775	9775
query75	2874	2768	2473	2473
query76	1863	1042	695	695
query77	399	403	312	312
query78	11236	11384	10756	10756
query79	2573	805	592	592
query80	1728	644	558	558
query81	571	284	253	253
query82	999	154	119	119
query83	332	293	243	243
query84	252	120	97	97
query85	891	483	431	431
query86	415	307	333	307
query87	3122	3084	3038	3038
query88	3502	2629	2649	2629
query89	418	373	347	347
query90	1966	170	166	166
query91	152	155	132	132
query92	77	78	68	68
query93	1117	837	508	508
query94	624	321	292	292
query95	579	395	317	317
query96	646	521	226	226
query97	2480	2493	2434	2434
query98	227	221	222	221
query99	995	978	940	940
Total cold run time: 258291 ms
Total hot run time: 184082 ms

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.53% (19581/37275)
Line Coverage 36.16% (182807/505489)
Region Coverage 32.51% (142015/436786)
Branch Coverage 33.47% (61557/183913)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.42% (26117/36567)
Line Coverage 54.24% (273825/504856)
Region Coverage 51.43% (227163/441719)
Branch Coverage 52.82% (97671/184928)

@xuchenhao
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 29224 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 323889bc722ed99cb7af91b6ab5451800b66a10e, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17657	4491	4309	4309
q2	q3	10644	804	516	516
q4	4678	359	256	256
q5	7575	1210	1020	1020
q6	182	173	150	150
q7	777	835	671	671
q8	9297	1472	1377	1377
q9	4841	4836	4701	4701
q10	6832	1865	1655	1655
q11	466	270	241	241
q12	709	574	478	478
q13	17767	4236	3464	3464
q14	233	235	215	215
q15	958	814	814	814
q16	775	749	694	694
q17	724	913	441	441
q18	6012	5359	5298	5298
q19	1118	984	639	639
q20	510	495	391	391
q21	4616	2099	1620	1620
q22	391	360	274	274
Total cold run time: 96762 ms
Total hot run time: 29224 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4684	4584	4588	4584
q2	q3	1797	2232	1824	1824
q4	926	1181	776	776
q5	4058	4355	4348	4348
q6	190	181	149	149
q7	1783	1669	1547	1547
q8	2492	2728	2635	2635
q9	7678	7458	7462	7458
q10	2819	2833	2415	2415
q11	521	426	418	418
q12	522	592	549	549
q13	4014	4531	3715	3715
q14	293	312	292	292
q15	873	814	790	790
q16	715	755	740	740
q17	1217	1618	1340	1340
q18	7160	6997	6562	6562
q19	948	919	950	919
q20	2101	2150	2023	2023
q21	4039	3545	3455	3455
q22	467	452	401	401
Total cold run time: 49297 ms
Total hot run time: 46940 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185085 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 323889bc722ed99cb7af91b6ab5451800b66a10e, data reload: false

query5	4362	650	494	494
query6	319	220	195	195
query7	4228	476	278	278
query8	343	249	265	249
query9	8709	2762	2748	2748
query10	507	388	334	334
query11	17024	16918	16655	16655
query12	180	126	127	126
query13	1240	477	374	374
query14	6011	3245	3086	3086
query14_1	2868	2838	2810	2810
query15	204	192	177	177
query16	978	483	475	475
query17	1117	722	628	628
query18	2451	456	357	357
query19	217	214	197	197
query20	138	133	134	133
query21	227	148	123	123
query22	5126	6173	5411	5411
query23	17541	17221	17139	17139
query23_1	16990	17056	16946	16946
query24	7111	1622	1240	1240
query24_1	1222	1250	1229	1229
query25	554	455	396	396
query26	1229	274	150	150
query27	2767	477	282	282
query28	4492	1899	1876	1876
query29	832	565	473	473
query30	312	236	208	208
query31	863	726	654	654
query32	88	71	70	70
query33	513	339	269	269
query34	919	920	571	571
query35	637	675	592	592
query36	1092	1091	963	963
query37	135	96	83	83
query38	2968	2904	2928	2904
query39	891	865	823	823
query39_1	822	837	834	834
query40	230	152	137	137
query41	61	60	61	60
query42	114	103	100	100
query43	396	391	357	357
query44	
query45	205	189	180	180
query46	911	996	630	630
query47	2116	2098	2044	2044
query48	316	319	241	241
query49	632	469	407	407
query50	699	291	213	213
query51	4184	4106	4123	4106
query52	107	109	98	98
query53	292	335	290	290
query54	304	291	256	256
query55	93	87	84	84
query56	310	323	313	313
query57	1396	1335	1286	1286
query58	284	273	282	273
query59	2602	2725	2526	2526
query60	347	328	332	328
query61	150	148	149	148
query62	595	589	550	550
query63	312	284	280	280
query64	4976	1294	1005	1005
query65	
query66	1469	467	351	351
query67	16416	16397	16223	16223
query68	
query69	394	301	277	277
query70	1007	941	984	941
query71	344	309	302	302
query72	2775	2778	2551	2551
query73	553	576	339	339
query74	10022	9946	9780	9780
query75	2881	2766	2484	2484
query76	2283	1044	710	710
query77	382	405	330	330
query78	11212	11412	10685	10685
query79	2452	830	628	628
query80	1742	658	574	574
query81	573	317	247	247
query82	1008	152	117	117
query83	325	272	248	248
query84	249	122	92	92
query85	905	481	445	445
query86	408	304	290	290
query87	3126	3101	3083	3083
query88	3687	2705	2680	2680
query89	432	382	351	351
query90	1990	180	174	174
query91	162	159	136	136
query92	75	75	71	71
query93	1071	838	527	527
query94	637	320	302	302
query95	574	342	381	342
query96	652	544	230	230
query97	2477	2504	2399	2399
query98	227	214	216	214
query99	1010	988	932	932
Total cold run time: 254950 ms
Total hot run time: 185085 ms

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.56% (19625/37337)
Line Coverage 36.18% (183216/506359)
Region Coverage 32.50% (142219/437574)
Branch Coverage 33.45% (61667/184364)

@xuchenhao
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 28688 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d69c2f14c8f9e406330a1411f7664181fdfcc03b, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17624	4494	4288	4288
q2	q3	10646	766	522	522
q4	4676	352	255	255
q5	7541	1177	1024	1024
q6	176	174	152	152
q7	774	840	670	670
q8	9641	1452	1318	1318
q9	4877	4672	4640	4640
q10	6823	1877	1646	1646
q11	451	250	250	250
q12	742	564	497	497
q13	17807	4274	3429	3429
q14	227	240	215	215
q15	963	795	788	788
q16	765	711	671	671
q17	698	887	419	419
q18	6326	5331	5248	5248
q19	1338	968	597	597
q20	511	493	397	397
q21	4488	1832	1417	1417
q22	341	291	245	245
Total cold run time: 97435 ms
Total hot run time: 28688 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4449	4339	4325	4325
q2	q3	1751	2153	1715	1715
q4	842	1159	757	757
q5	4024	4340	4278	4278
q6	185	170	145	145
q7	1699	1600	1487	1487
q8	2382	2649	2505	2505
q9	7475	7539	7418	7418
q10	2668	2813	2427	2427
q11	505	435	423	423
q12	522	602	472	472
q13	3987	4465	3712	3712
q14	293	309	277	277
q15	864	835	805	805
q16	714	769	712	712
q17	1200	1562	1311	1311
q18	7149	6892	6546	6546
q19	927	898	943	898
q20	2053	2211	1990	1990
q21	3913	3626	3333	3333
q22	501	433	374	374
Total cold run time: 48103 ms
Total hot run time: 45910 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184282 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d69c2f14c8f9e406330a1411f7664181fdfcc03b, data reload: false

query5	4469	653	535	535
query6	333	227	202	202
query7	4218	461	271	271
query8	332	244	233	233
query9	8741	2793	2767	2767
query10	526	377	341	341
query11	17029	17484	17386	17386
query12	209	146	147	146
query13	1491	491	362	362
query14	6766	3282	3169	3169
query14_1	2954	2961	2978	2961
query15	216	195	181	181
query16	1009	482	470	470
query17	1493	772	643	643
query18	3425	498	331	331
query19	202	207	179	179
query20	143	127	133	127
query21	210	134	116	116
query22	5964	5650	4943	4943
query23	17190	16817	16526	16526
query23_1	16875	16599	16773	16599
query24	7191	1613	1220	1220
query24_1	1243	1226	1226	1226
query25	540	446	406	406
query26	1237	256	160	160
query27	2746	465	284	284
query28	4463	1880	1901	1880
query29	837	550	480	480
query30	314	246	212	212
query31	888	715	642	642
query32	88	70	72	70
query33	521	350	284	284
query34	910	923	562	562
query35	629	666	597	597
query36	1085	1108	979	979
query37	131	91	81	81
query38	2994	2970	2864	2864
query39	882	872	869	869
query39_1	813	802	838	802
query40	227	144	136	136
query41	62	62	56	56
query42	113	99	101	99
query43	369	381	354	354
query44	
query45	192	191	187	187
query46	891	992	597	597
query47	2130	2133	2077	2077
query48	301	326	231	231
query49	625	455	398	398
query50	678	276	216	216
query51	4045	4106	4086	4086
query52	104	105	96	96
query53	289	337	291	291
query54	296	262	263	262
query55	91	82	83	82
query56	313	310	311	310
query57	1350	1335	1277	1277
query58	284	283	273	273
query59	2588	2606	2504	2504
query60	341	336	326	326
query61	147	150	147	147
query62	628	610	548	548
query63	314	288	277	277
query64	4832	1240	990	990
query65	
query66	1390	459	350	350
query67	16288	16339	16236	16236
query68	
query69	398	306	288	288
query70	959	958	914	914
query71	341	303	297	297
query72	2922	2821	2569	2569
query73	556	569	338	338
query74	9977	9896	9774	9774
query75	2855	2734	2482	2482
query76	2297	1043	687	687
query77	359	377	321	321
query78	11186	11334	10710	10710
query79	2648	801	612	612
query80	1843	610	547	547
query81	562	271	250	250
query82	980	148	113	113
query83	328	261	246	246
query84	244	117	98	98
query85	882	469	421	421
query86	415	329	306	306
query87	3105	3087	3004	3004
query88	3656	2682	2656	2656
query89	423	368	340	340
query90	2016	180	178	178
query91	167	151	135	135
query92	77	70	80	70
query93	1189	814	508	508
query94	651	309	283	283
query95	582	398	315	315
query96	632	517	230	230
query97	2472	2496	2450	2450
query98	231	223	218	218
query99	1028	998	901	901
Total cold run time: 257688 ms
Total hot run time: 184282 ms

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.58% (19630/37337)
Line Coverage 36.22% (183387/506359)
Region Coverage 32.50% (142219/437574)
Branch Coverage 33.45% (61672/184364)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.51% (26144/36559)
Line Coverage 54.26% (273924/504806)
Region Coverage 51.49% (227417/441711)
Branch Coverage 52.88% (97784/184928)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Spill GC crashes due to missing pointer dereference in LocalFileSystem::list_impl

3 participants