[druid-user] Druid (0.14.2) kill task stuck - ReadTimeoutException

Hi Team,

We are seeing an issue with running of kill task on Druid 0.14.2, which is erroring out with ReadTimeoutException. Below are the kill task logs.

This is happening only for one of the datasource on Druid cluster. The kill task works without any such errors for other data sources on the same cluster.

Also, verified if the Overlord node is reachable from Middle Manager using ‘curl http://10.x.x.x:8090/status

2021-05-04T05:36:35,971 WARN [HttpClient-Netty-Worker-2] org.apache.druid.java.util.http.client.pool.ResourcePool - Resource at key[10.xxx.xxx.xxx:8090] was returned multiple times?
2021-05-04T05:36:35,972 WARN [task-runner-0-priority-0] org.apache.druid.discovery.DruidLeaderClient - Request[http://10.xxx.xxx.xxx:8090/druid/indexer/v1/action] failed.
org.jboss.netty.handler.timeout.ReadTimeoutException
at org.jboss.netty.handler.timeout.ReadTimeoutHandler.(ReadTimeoutHandler.java:84) ~[netty-3.10.6.Final.jar:?]
at org.apache.druid.java.util.http.client.NettyHttpClient.go(NettyHttpClient.java:172) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.http.client.AbstractHttpClient.go(AbstractHttpClient.java:33) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.discovery.DruidLeaderClient.go(DruidLeaderClient.java:174) ~[druid-server-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.discovery.DruidLeaderClient.go(DruidLeaderClient.java:144) ~[druid-server-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.indexing.common.actions.RemoteTaskActionClient.submit(RemoteTaskActionClient.java:79) ~[druid-indexing-service-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.indexing.common.task.AbstractFixedIntervalTask.isReady(AbstractFixedIntervalTask.java:75) ~[druid-indexing-service-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.indexing.worker.executor.ExecutorLifecycle.start(ExecutorLifecycle.java:171) ~[druid-indexing-service-0.14.2-incubating.jar:0.14.2-incubating]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_252]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_252]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_252]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
at org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:427) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.java.util.common.lifecycle.Lifecycle.start(Lifecycle.java:323) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.guice.LifecycleModule$2.start(LifecycleModule.java:138) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:107) ~[druid-services-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.cli.CliPeon.run(CliPeon.java:356) ~[druid-services-0.14.2-incubating.jar:0.14.2-incubating]
at org.apache.druid.cli.Main.main(Main.java:118) ~[druid-services-0.14.2-incubating.jar:0.14.2-incubating]

Could someone please guide me on resolving this?

Thank you in advance.

Regards,
Vinay Patil