Issues with hdfs as deepstorage

I set up an indexing service using hdfs as deep storage but failed to output anything to hdfs, instead task logs and segments all went to local storage.

Here is what I did:

In _common/


In middleManager’s


In druid startup script, compose the classpath as

DRUID_CP={SCRIPT_DIR}/config/_common DRUID_CP={DRUID_CP}:{SCRIPT_DIR}/config/middelManager|overlord|historical... DRUID_CP={DRUID_CP}:{SCRIPT_DIR}/lib/* DRUID_CP={DRUID_CP}:hadoop classpath

I followed the suggestion by!searchin/druid-user/hdfs$20deepstorage$20/druid-user/kQMzQpf9ZXY/QSdPiqELMz8J.

Hi - I think it should be:

Instead of


Thank you for pointing out. Yes it is the culprit. The segment data started to show in the hdfs now.

One more question, do you know why the task logs did not go to hdfs, despite the setting:


In the middleManager’s

Task logs are uploaded to HDFS when the task completes. When you say the task is not uploaded to HDFS, do you mean after the task completes you cannot see the log in hdfs? That config looks fine to me.