Logs are missing

Hello,

I started druid Cluster by using below command

nohup java cat conf/druid/middleManager/jvm.config | xargs -cp conf/druid/_common:conf/druid/middleManager:lib/* org.apache.druid.cli.Main server middleManager >> /home/app/logs/druid/mm.log &

now the cluster is up and running i can able to load the Overlord and Coordinator pages in my browser

when i start trying batch ingestion with sample data it is FAILED

./post-index-task --file …/quickstart/tutorial/wikipedia-index.json

Beginning indexing data for wikipedia

Task started: index_wikipedia_2019-06-03T19:04:37.075Z

Task log: http://localhost:8090/druid/indexer/v1/task/index_wikipedia_2019-06-03T19:04:37.075Z/log

Task status: http://localhost:8090/druid/indexer/v1/task/index_wikipedia_2019-06-03T19:04:37.075Z/status

Task index_wikipedia_2019-06-03T19:04:37.075Z still running…

Task finished with status: FAILED

To debug it is not generating logs


No log was found for this task. The task may not exist, or it may not have begun running yet.


in my /var "druid" folder is not available i just created and given the full 777 permission to load the logs

even though logs are not generating

Appreciate your help






Hi Lee,
Assuming all the services(coordinator, overlord, historical, middleManager and broker) are up and running properly,

what is your deep storage type(local or HDFS or s3)?

Did you create the paths for deep storage and indexing-logs and do they have proper permissions?

Are you seeing logs in the following paths ?

–druid_home_folder/var/sv/overlord/current
–druid_home_folder/var/sv/middleManager/current

if yes, do you see any errors in those logs?

Thanks,

–siva

Hi Siva,

Thanks a lot for your time to reply

I did not see any logs in the --druid_home_folder/var

in my --druid_home_folder/var directory I have only derby.log and metadata.db

And regarding permissions, I have full permission to the folder

Am I miss anything?

Regards,

Moulali

Hi Moulali,
Is it single node cluster or multi-node cluster?
If it is multi-node cluster, is your deep storage HDFS or S3?

Did you create the paths for deep storage and indexing-logs in HDFS or S3 if your deep storage HDFS or S3?

Thank you.

–siva

Hi Siva,

Thanks for your reply,

i recreated the cluster and i am able to see the logs /var/druid/index-logs at Middle Manager Node folder but in the overlord dashboard logs are not available

No log was found for this task. The task may not exist, or it may not have begun running yet.

Please find my below cluster setup

2 nodes (MM and Overlord, ZK, ) and Derby in one of the node
1 Node (ZK, hisorical)
1 Node (Historical)
2 Nodes (Middle Manager)
2 Nodes (Brokers)

Regards,

Lee

Have you checked the Overlord logs on the node(s) on which it is set up ?