In middle manager node getting IOException: Stream closed and task completed with FAILED status

Hi all,

I am using

  1. Durid version is: druid-0.12.0

  2. Kafaka version: kafka_2.11-1.1.0

  3. druid-kafka-indexing-service extension for ingestion of kafka steam into druid.

But in middle manager node I am getting following error for every task and task status completed with failed status. I am also attaching here full task index log if that helps to get insight.

2018-04-19T06:09:26,225 INFO [qtp1680365850-23] io.druid.indexing.overlord.ForkingTaskRunner - Killing process for task: index_kafka_test_fcb3a164d8ca43e_heaojnbg

2018-04-19T06:09:26,512 INFO [forking-task-runner-0] io.druid.indexing.common.tasklogs.FileTaskLogs - Wrote task log to: /tmp/druid-logs/index_kafka_test_fcb3a164d8ca43e_heaojnbg.log

2018-04-19T06:09:26,631 INFO [qtp1680365850-25] io.druid.indexing.overlord.ForkingTaskRunner - Killing process for task: index_kafka_test_fcb3a164d8ca43e_heaojnbg

2018-04-19T06:09:26,547 INFO [forking-task-runner-0] io.druid.indexing.overlord.ForkingTaskRunner - Exception caught during execution

java.io.IOException: Stream closed

at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170) ~[?:1.8.0_161]

at java.io.BufferedInputStream.read1(BufferedInputStream.java:291) ~[?:1.8.0_161]

at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_161]

at java.io.FilterInputStream.read(FilterInputStream.java:107) ~[?:1.8.0_161]

at com.google.common.io.ByteStreams.copy(ByteStreams.java:175) ~[guava-16.0.1.jar:?]

at io.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:452) [druid-indexing-service-0.12.0.jar:0.12.0]

at io.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:224) [druid-indexing-service-0.12.0.jar:0.12.0]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_161]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]

2018-04-19T06:09:26,822 INFO [forking-task-runner-0] io.druid.indexing.overlord.ForkingTaskRunner - Removing task directory: var/druid/task/index_kafka_test_fcb3a164d8ca43e_heaojnbg

2018-04-19T06:09:26,822 INFO [qtp1680365850-28] io.druid.indexing.overlord.ForkingTaskRunner - Ignoring request to cancel unknown task: index_kafka_test_fcb3a164d8ca43e_heaojnbg

2018-04-19T06:09:26,894 INFO [qtp1680365850-29] io.druid.indexing.overlord.ForkingTaskRunner - Ignoring request to cancel unknown task: index_kafka_test_fcb3a164d8ca43e_heaojnbg

2018-04-19T06:09:27,196 INFO [WorkerTaskMonitor] io.druid.indexing.worker.WorkerTaskMonitor - Job’s finished. Completed [index_kafka_test_fcb3a164d8ca43e_heaojnbg] with status [FAILED]

Highly appreciate your help on this.

Thanks

Sohel

index_kafka_test_fcb3a164d8ca43e_heaojnbg.log (1020 KB)

same problem