Failure during persist and merge

ankur_jain · June 11, 2020, 9:46pm

hi All,

I am seeing the following error during handing over segments to hdfs. I have to replicas and I noticed that only one replica fails. The other is always successful. Also, this error is intermittent and does not happen always.

I looked through the open issues in github and there were similar(not exactly the same) issues reported because of a bug in hdfs-storage extension but in older versions of druid.

I am using version 0.16 incubating.

Appreciate your help and insight.

ankur_jain · June 24, 2020, 3:04am

Anyone?

navis94 · June 24, 2020, 5:16am

Could you check name node log containg 'FSDirectory.unprotectedRenameTo'?

gaurav · June 24, 2020, 7:05am

You are using viewfs://. doesn’t look like it supports all the methods needed for Druid to function. It is failing when trying to rename “at org.apache.hadoop.fs.FileSystem.rename(FileSystem.java:1331)”. Use only prescribed file systems for deep storage.