Batch data ingestion logs

I ingest batch data from hdfs, everything is ok, but we can’t see the logs of all batch tasks while streaming task logs are complete. there is only log message are below:

No log was found for this task. The task may not exist, or it may not have begun running yet.

but the task has been SUCCESS and segment is already ready. What should I do to find the previous logs?

Sounds like there might be problems uploading the task log to the configured log storage, you could check the location specified by druid.indexer.logs.* config to see if the task log was successfully written

yep, we configured the index-logs location to the hdfs cluster, there are exactly all kafka index service task logs in indexing-logs , but no batch logs.

在 2018年7月17日星期二 UTC+8上午8:59:06,Jonathan Wei写道:

Frank, reviewing the overlord and middlemanager logs, maybe you can discover if some problem or error is happing when the batch logs are trying to generate.

yep, surprised, the logs are there when the tasks are running, however, once the task has finished, the logs are gone. I think there are something wrong about persisting the index-logs

在 2018年7月18日星期三 UTC+8上午12:48:16,jimenarp写道: