Druid Cluster Ingestion Failing

Hi,

I’m trying to setup a druid cluster on aws ec2.

I have the following configuration-

Master Instance -> Coordinatior, Overlord, Zookeeper

Data Instance -> Historical, Middlemanager

Query Instance -> Broker

Deep Storage -> S3

Metadata -> Derby

I’m trying to ingest the wikipedia tutorial data for sanity checks purpose using the following method.

  1. SSH into Master Node

  2. cd druid

  3. curl -X ‘POST’ -H ‘Content-Type:application/json’ -d @quickstart/tutorial/wikipedia-index.json http://:8081/druid/indexer/v1/task

However, the task always fails with the below error.

2019-08-02T16:55:48,425 WARN [HttpClient-Netty-Boss-0] org.jboss.netty.channel.SimpleChannelUpstreamHandler - EXCEPTION, please implement org.jboss.netty.handler.codec.http.HttpContentDecompressor.exceptionCaught() for proper handling.

java.net.ConnectException: Connection refused: localhost/127.0.0.1:8081

at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_222]

at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_222]

at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152) ~[netty-3.10.6.Final.jar:?]

at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) [netty-3.10.6.Final.jar:?]

at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) [netty-3.10.6.Final.jar:?]

at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) [netty-3.10.6.Final.jar:?]

at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) [netty-3.10.6.Final.jar:?]

at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.10.6.Final.jar:?]

at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.10.6.Final.jar:?]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]

I have done the following zookeeper setup on my master

–> conf/zk/zoo.cfg

–> server.1=:2888:3888

I have also added the below lines in the common.runtime.properties of all of my instances.

–> druid.zk.service.host=

Am I missing some setup? If anybody can help, it would be great.

What should be the value of druid.host on all of my instances. It is still localhost as nothing was mentioned in the guide about changing it.

I’m following the steps here : https://druid.apache.org/docs/latest/tutorials/cluster.html#fresh-deployment

Regards,

Darshan

Can you please forward your common.runtime.properties? Is the common.runtime.properties the same for every node? Also - make sure the coordinator process is running on your master node.

How are you starting the processes per instance?

Eric Graham

Solutions Engineer -** **Imply

**cell: **303-589-4581

email: eric.graham@imply.io

www.imply.io