标签：task iceberg java lock HiveTableOperations Waiting Tasks apache org

执行spark任务居然碰到了hive中表的被锁了，导致写数据时失败，这个还真是头一次碰到过这回事。

错误信息如下：

22/09/04 21:02:28 WARN Tasks: Retrying task after failure: Waiting for lock.
org.apache.iceberg.hive.HiveTableOperations$WaitingForLockException: Waiting for lock.
        at org.apache.iceberg.hive.HiveTableOperations.lambda$acquireLock$9(HiveTableOperations.java:460)
        at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:404)
        at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)
        at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)
        at org.apache.iceberg.hive.HiveTableOperations.acquireLock(HiveTableOperations.java:454)
        at org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:226)
        at org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:127)
        at org.apache.iceberg.SnapshotProducer.lambda$commit$2(SnapshotProducer.java:304)
        at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:404)
        at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)
        at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)
        at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:190)
        at org.apache.iceberg.SnapshotProducer.commit(SnapshotProducer.java:282)
        at org.apache.iceberg.BaseReplacePartitions.commit(BaseReplacePartitions.java:26)
        at org.apache.iceberg.spark.source.Writer.commitOperation(Writer.java:144)
        at org.apache.iceberg.spark.source.Writer.replacePartitions(Writer.java:181)
        at org.apache.iceberg.spark.source.Writer.commit(Writer.java:119)
        at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec.doExecute(WriteToDataSourceV2Exec.scala:76)
 

跑到hive的元数据库中查询时，在表HIVE_LOCKS中找到了一些信息。

根据出错信息，根据hl_table找到相关的表，查看最近的心跳时间字段，hl_last_heartbeat。
删除对应的记录即可。

delete  from HIVE_LOCKS where hl_table = '出错的表名';

标签：task,iceberg,java,lock,HiveTableOperations,Waiting,Tasks,apache,org
来源： https://www.cnblogs.com/30go/p/16656135.html

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。