OverlappingFileLockException during compaction

Hi all,

I am using Druid 0.16-incubating. When I manually run compaction job, a lot of them failed due to java.nio.channels.OverlappingFileLockException. There shouldn’t be any late data since data being compacted is from two month ago and the data source should drop any late data at that time. What might raise the exceptions and how to avoid them?

Thanks!

Ruibin

Hi!

Is it possible to upgrade to a newer version of Druid?

OverlappingFileLockException is not thrown in Druid, but in the JVM when the same file (or overlapping region of a file) is locked more than once by the same JVM or threads. This exception is never supposed to be thrown, and so it is likely a bug if it occurred. I have never seen this error before, and am not sure whether or not the compaction task does anything with this error. Rather, this error would be likely thrown with any task type if it’s a legit bug. To narrow down the root cause of the potential bug, we need more information such as full stack trace of the exception, how your cluster was configured (especially if you are using the Indexer or not), and the full spec and the log of the task where the exception was thrown.