missing data with multiple realtime servers

Hi all,

I have two kafka servers with one topic with two partitions. There are two realtime nodes. I get only half of data when both nodes are running. All data appears when I disable one realtime node. It’s 0.8.

Broker knows about both servers

@400000005730d536394f45b4 2016-05-09T18:21:32,961 INFO [qtp1033569098-81] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://m3d9.int:8084

@400000005730d537074561ac 2016-05-09T18:21:33,121 INFO [qtp1033569098-81] com.metamx.http.client.pool.ChannelResourceFactory - Generating: http://m3d10.int:8084

@400000005730d69134e7ffac 2016-05-09T18:27:19,887 INFO [ServerInventoryView-0] io.druid.client.BatchServerInventoryView - Server Disappeared[DruidServerMetadata{name=‘m3d10.int:8084’, host=‘m3d10.int:8084’, maxSize=0, tier=’_d

efault_tier’, type=‘realtime’, priority=‘0’}]

Any clues?

Thanks in advance.


Hi Alexey,

Sounds like you need to configure your realtime nodes to have different partition numbers in the shard spec, see: http://druid.io/docs/latest/ingestion/stream-pull.html#sharding.

(I’m assuming that you have two realtime nodes for the purpose of scaling out your ingestion; if you’re trying to do replication, then make sure that both nodes have the same Kafka [group.id].)

Hey Dave,

thanks for your answer.

I’m trying to replicate. I do have same group.id

   "ioConfig": {

        "firehose": {

            "consumerProps": {

                "auto.commit.enable": "false",

                "auto.offset.reset": "largest",

                "fetch.message.max.bytes": "1048586",

                "group.id": "vpaid-group",

                "zookeeper.connect": "zk1.int:2181",

                "zookeeper.connection.timeout.ms": "15000",

                "zookeeper.session.timeout.ms": "15000",

                "zookeeper.sync.time.ms": "5000"


            "feed": "vpaid",

            "type": "kafka-0.8"


        "plumber": {

            "type": "realtime"


        "type": "realtime"


Hi Alexey,
Since you are trying to replicate, you need to have have multiple nodes with same druid partition number and different consumer kafka consumer group.id, so that each of the realtime nodes consumes all the data from kafka.

Sorry, I typed that wrong; Nishant is right: if you want each of your consumers to get a full set of messages from the Kafka broker, they need to have different group ids. Sorry for the confusion.