Hadoop indexing task fails with -- Conflicting property name definitions: 'maxRowsInMemory' error

Hello All,

I have been trying to get druid indexing to work with Hortonworks Data Platform 2.4. Based on the errors encountered during my workout, i had to create a new build for druid based on instructions available online. My current druid config is – druid 0.9.2, hadoop-client 2.7.1, jackson 2.3.1 and guice 3.0; rest other components are with out of box config.

I was able to get the binaries placed on my RHEL server and get all the druid services started without any errors. However, when i try to run a indexing job i am getting the below mentioned error( i am trying to run the quickstart/json spec to perform the test indexing task):

task ---- curl -X ‘POST’ -H ‘Content-Type:application/json’ -d @quickstart/wikiticker-index.json http://:8090/druid/indexer/v1/task

Error —

Error 500

HTTP ERROR: 500

Problem accessing /druid/indexer/v1/task. Reason:

    java.lang.IllegalStateException: Conflicting property name definitions: 'maxRowsInMemory' (for [method io.druid.indexer.HadoopTuningConfig#getRowFlushBoundary(0 params)]) vs 'rowFlushBoundary' (for [parameter #13, annotations: {interface com.fasterxml.jackson.annotation.JsonProperty=@com.fasterxml.jackson.annotation.JsonProperty(value=rowFlushBoundary, required=false)}])

Powered by Jetty://

Spec file modification tried to overcome this error:

“tuningConfig” : {

“type” : “hadoop”,

“rowFlushBoundary” : 0,

“partitionsSpec” : {

“type” : “hashed”,

“targetPartitionSize” : 5000000

},

“jobProperties” : {}

}

Any help is greatly appreciated.

Thanks

Chandra.

Hi Chandra, can you rename rowFlushBoundary to maxRowsInMemory and see if that helps?

Also, you might be interested to know we updated our documentation recently about CDH and HDP: https://github.com/druid-io/druid/blob/master/docs/content/operations/other-hadoop.md

Hi, have you found a solution for this? After compiling Druid with jackson version 2.3.5 and trying to index I got the same error message.

Would you share your index task spec?

2017년 6월 27일 (화) 오전 12:17, Tomas Farkas funes79@gmail.com님이 작성: