-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
Search before asking
- I searched the issues and found no similar issues.
Linkis Component
linkis-engineplugin-jdbc
What happened
English:
When executing StarRocks tasks through JDBC EngineConn, if the task contains multiple SQL statements (such as DROP TABLE and CREATE TABLE AS SELECT), canceling the task after a long execution time results in inconsistent error states.
Problem Scenario:
DROP TABLE IF EXISTS old_table;
CREATE TABLE new_table AS SELECT * FROM source_table WHERE condition;When a user cancels a running task:
- System reports error: Table already exists
- But querying the table returns error: Table not found
This indicates that there is a problem with transaction handling when canceling the task, resulting in inconsistency between table metadata and actual data.
中文:
通过JDBC EngineConn执行StarRocks任务时,当任务包含多条SQL语句(如 DROP TABLE 和 CREATE TABLE AS SELECT),在任务执行过长时间后用户取消任务会出现不一致的错误状态。
问题场景:
DROP TABLE IF EXISTS old_table;
CREATE TABLE new_table AS SELECT * FROM source_table WHERE condition;当用户取消正在执行的任务时:
- 系统提示错误:表文件已经存在(Table already exists)
- 但查询该表时又报错:表不存在(Table not found)
这表明任务取消时的事务处理存在问题,导致表的元数据和实际数据不一致。
What you expected to happen
English:
When a task is canceled, the system should:
- Properly rollback executed statements or ensure transaction integrity
- Avoid contradictory states of "table exists" and "table not found"
- Provide clear task cancellation status feedback to users
- Clean up intermediate states to avoid data inconsistency
中文:
任务取消时应该:
- 正确回滚已执行的语句或保证事务完整性
- 不出现"表已存在"和"表不存在"的矛盾状态
- 给用户明确的任务取消状态反馈
- 清理中间状态,避免数据不一致
How to reproduce
English:
- Configure JDBC engine to connect to StarRocks database
- Submit SQL task with multiple statements:
DROP TABLE IF EXISTS test_table;
CREATE TABLE test_table AS SELECT * FROM large_table WHERE id > 1000000;- Wait for CREATE TABLE statement to start executing (ensure there is a large amount of data to process)
- Click cancel during task execution
- Observe error messages and table status
Expected Result:
- Query test_table: Table not found
Actual Result:
- Error on cancellation: Table already exists
- Error on query: Table not found
中文:
- 配置JDBC引擎连接StarRocks数据库
- 提交包含多条语句的SQL任务:
DROP TABLE IF EXISTS test_table;
CREATE TABLE test_table AS SELECT * FROM large_table WHERE id > 1000000;- 等待CREATE TABLE语句开始执行(确保有大量数据需要处理)
- 在任务执行过程中点击取消
- 观察错误信息和表状态
预期结果:
- 查询 test_table:Table not found(表不存在)
实际结果:
- 取消时报错:Table already exists
- 查询时报错:Table not found
Anything else
English:
Possible Causes:
- JDBC engine does not use transactions or transaction isolation level is inappropriate when executing multiple statements
- Incorrect timing of Statement.cancel() call when canceling task
- CREATE TABLE AS SELECT is a DDL+DML mixed operation, improper handling when canceled
- StarRocks table metadata and data file cleanup are not synchronized
Suggested Solutions:
- Check JDBC engine's task cancellation logic to ensure Statement.cancel() is called correctly
- Consider adding cleanup logic for DDL operations
- Add transaction support for multi-statement tasks (if database supports it)
- Add status check and cleanup mechanism after task cancellation
- For CREATE TABLE AS SELECT type statements, try DROP TABLE cleanup when canceled
Impact Scope:
- Not only affects StarRocks, similar issues may exist for other JDBC data sources (such as ClickHouse, Doris, etc.)
- Consider generality when fixing
中文:
可能的原因:
- JDBC引擎执行多语句时没有使用事务或事务隔离级别不当
- 任务取消时Statement.cancel()调用时机不正确
- CREATE TABLE AS SELECT是DDL+DML混合操作,取消时处理不当
- StarRocks的表元数据和数据文件清理不同步
建议的解决方案:
- 检查JDBC引擎的任务取消逻辑,确保正确调用Statement.cancel()
- 对于DDL操作,考虑添加清理逻辑
- 为多语句任务添加事务支持(如果数据库支持)
- 增加任务取消后的状态检查和清理机制
- 针对CREATE TABLE AS SELECT类型的语句,在取消时尝试DROP TABLE清理
影响范围:
- 不仅影响StarRocks,可能对其他JDBC数据源(如ClickHouse、Doris等)也存在类似问题
- 建议在修复时考虑通用性
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Linkis Version
1.8.0
Java Version
OpenJDK 1.8.0
Additional Information
English:
- JDBC Statement.cancel() method
- Database transactions and isolation levels
- Atomicity of DDL statements
- StarRocks table creation and deletion mechanism
中文:
相关技术要点:
- JDBC Statement.cancel() 方法
- 数据库事务和隔离级别
- DDL语句的原子性
- StarRocks的表创建和删除机制