Plz help me for realtime node failed in 0.10.1

Hi all,
the error message:

INFO: Binding com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider to GuiceManagedComponentProvider with the scope “Singleton”

Sep 02, 2017 4:37:27 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider

INFO: Binding com.fasterxml.jackson.jaxrs.smile.JacksonSmileProvider to GuiceManagedComponentProvider with the scope “Singleton”

2017-09-02T04:37:27,592 ERROR [chief-0] io.druid.segment.realtime.RealtimeManager - [NullPointerException] aborted realtime processing[test]: {class=io.druid.segment.realtime.RealtimeManager, exceptionType=class java.lang.NullPointerException, exceptionMessage=null}

java.lang.NullPointerException

at kafka.utils.VerifiableProperties.containsKey(VerifiableProperties.scala:32) ~[?:?]

at kafka.utils.VerifiableProperties.getString(VerifiableProperties.scala:177) ~[?:?]

at kafka.utils.ZKConfig.(ZkUtils.scala:740) ~[?:?]

at kafka.consumer.ConsumerConfig.(ConsumerConfig.scala:90) ~[?:?]

at kafka.consumer.ConsumerConfig.(ConsumerConfig.scala:94) ~[?:?]

at io.druid.firehose.kafka.KafkaEightFirehoseFactory.connect(KafkaEightFirehoseFactory.java:89) ~[?:?]

at io.druid.firehose.kafka.KafkaEightFirehoseFactory.connect(KafkaEightFirehoseFactory.java:50) ~[?:?]

at io.druid.segment.realtime.FireDepartment.connect(FireDepartment.java:104) ~[druid-server-0.10.1.jar:0.10.1]

at io.druid.segment.realtime.RealtimeManager$FireChief.initFirehose(RealtimeManager.java:245) ~[druid-server-0.10.1.jar:0.10.1]

at io.druid.segment.realtime.RealtimeManager$FireChief.run(RealtimeManager.java:299) [druid-server-0.10.1.jar:0.10.1]

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_51]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_51]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_51]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_51]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_51]

realtime node runtime.properties

druid.service=druid/realtime

druid.host=10.90.35.43

druid.port=8084

Processing threads and buffers

druid.processing.buffer.sizeBytes=53687091

druid.processing.numThreads=7

druid.segmentCache.locations=[{“path”:“var/druid/segment-cache”,“maxSize”:130000000000}]

Query cache

druid.realtime.cache.useCache=false

druid.realtime.cache.populateCache=false

Operation

druid.realtime.specFile=/data/druid-0.10.1/conf/druid/realtime/ax.json

``

ax.json

[

{

“dataSchema”: {

“dataSource”: “test”,

“granularitySpec”: {

“type”: “uniform”,

“segmentGranularity”: “hour”,

“queryGranularity”: “minute”

},

“metricsSpec”: ,

“parser”: {

“parseSpec”: {

“dimensionsSpec”: {

“dimensionExclusions”: ,

“dimensions”: [

“ip”,

“uid”,

“ap”,

“tp”,

“bid”,

“rid”,

“cid”,

“area”,

“dm”,

“rb”,

“rc”,

“f”,

“osv”,

“st”,

“appv”

],

“spatialDimensions”:

},

“format”: “csv”,

“lostDelimiter”: “|@”,

“timestampSpec”: {

“column”: “times”,

“format”: “auto”

},

“columns”: [

“times”,

“ip”,

“uid”,

“ap”,

“tp”,

“bid”,

“rid”,

“cid”,

“area”,

“dm”,

“rb”,

“rc”,

“f”,

“osv”,

“st”,

“appv”

]

},

“type”: “string”

}

},

“ioConfig”: {

“type”: “realtime”,

“firehose”: {

“type”: “kafka-0.8”,

“feed”: “test”,

“comsumerProps”: {

“auto.commit.enable”: “false”,

“auto.offset.reset”: “largest”,

“fetch.message.max.bytes”: “1048586”,

“group.id”: “ax-log-consumer”,

“zookeeper.connect”: “10.50.11.47:2181,10.50.11.48:2181,10.50.11.49:2181,10.50.11.55:2181,10.50.11.56:2181”,

“zookeeper.connection.timeout.ms”: “15000”,

“zookeeper.session.timeout.ms”: “15000”,

“zookeeper.sync.time.ms”: “5000”

}

},

“plumber”: {

“type”: “realtime”

}

},

“tuningConfig”: {

“type”: “realtime”,

“maxRowsInMemory”: “100000”,

“intermediatePersistPeriod”: “PT10M”,

“windowPeriod”: “PT10M”,

“rejectionPolicy”: {

“type”: “messageTime”

},

“basePersistDirectory”: “/data/druid/realtime/basePersist”

}

}

]

``

thank’s

hi every body。Use debug i fount that

io.druid.segment.realtime.RealtimeManager$FireChief.getFirehoseFactoryV2()

io.druid.segment.realtime.RealtimeManager$FireChief.getFirehoseFactory()

both are null.

It looks like you misspelled “comsumerProps”, it should really be “consumerProps”.

hi Gina,
Thank you for your reply,
but i have consumerProps in firehose,

“ioConfig”: {

“type”: “realtime”,

“firehose”: {

“type”: “kafka-0.8”,

“feed”: “test”,

“comsumerProps”: {

“auto.commit.enable”: “false”,

“auto.offset.reset”: “largest”,

“fetch.message.max.bytes”: “1048586”,

“group.id”: “ax-log-consumer”,

“zookeeper.connect”: “10.50.11.47:2181,10.50.11.48:2181,10.50.11.49:2181,10.50.11.55:2181,10.50.11.56:2181”,

“zookeeper.connection.timeout.ms”: “15000”,

“zookeeper.session.timeout.ms”: “15000”,

“zookeeper.sync.time.ms”: “5000”

}

},

“plumber”: {

“type”: “realtime”

}

}

Where was it written incorrectly?

Best Regards.

在 2017年9月4日星期一 UTC+8上午5:47:26,Gian Merlino写道:

“comsumerProps” in that config is spelled with two "m"s but there should only be one.

Dear Gina, I’m sorry for my stupid mistake.

Thanks again for your help.

在 2017年9月5日星期二 UTC+8上午7:04:06,Gian Merlino写道:

No worries, hope you got it working now!