Unable to run batch ingestion example on cluster

I am attempting to setup druid-0.10.0 on an IBM BigInsights cluster (equivalent to HDP). After numerous attempts and tweaks, I can still not get the simple files-based tutorial (http://druid.io/docs/0.10.0/tutorials/tutorial-batch.html).

I have the Overlord, Coordinator, and Broker running on one server in my cluster and the Middle Manager and Historical running on a second server in the cluster and am using HDFS for deep storage. BigInsights ships with hadoop client 2.7.3, so I have tried different runs after setting

mapreduce.job.classloader = true

or

mapreduce.job.user.classpath.first = true

Here is my task:

{
“type” : “index_hadoop”,
“spec” : {
“ioConfig” : {
“type” : “hadoop”,
“inputSpec” : {
“type” : “static”,
“paths” : “pageviews.json”
}
},
“dataSchema” : {
“dataSource” : “pageviews”,
“granularitySpec” : {
“type” : “uniform”,
“segmentGranularity” : “day”,
“queryGranularity” : “none”,
“intervals” : [“2015-09-01/2015-09-02”]
},
“parser” : {
“type” : “hadoopyString”,
“parseSpec” : {
“format” : “json”,
“dimensionsSpec” : {
“dimensions” : [ “url”, “user”
]
},
“timestampSpec” : {
“format” : “auto”,
“column” : “time”
}
}
},
“metricsSpec” : [
{
“name” : “views”,
“type” : “count”
},
{
“name” : “latencyMs”,
“type” : “doubleSum”,
“fieldName” : “latencyMs”
}
]
},
“tuningConfig” : {
“type” : “hadoop”,
“partitionsSpec” : {
“type” : “hashed”,
“targetPartitionSize” : 5000000
},
“jobProperties” : {
“mapreduce.job.classloader”: “true”,
“mapreduce.map.java.opts”: “-Duser.timezone=UTC -Dfile.encoding=UTF-8”,
“mapreduce.reduce.java.opts”: “-Duser.timezone=UTC -Dfile.encoding=UTF-8”
},
“ignoreInvalidRows” : “true”,
“leaveIntermediate” : “true”
}
}
}

My common.runtime.properties

common.runtime.properties (3.64 KB)

log (163 KB)

my-index-task.json (1.43 KB)

Hi,Jeremy

you can check your mapreduce task log.

在 2017年8月17日星期四 UTC+8上午1:01:00,Jeremy写道: