Batch Druid Indexing on Hadoop

Hi,

I’m trying to get the command line hadoop indexer working in 0.8.2.

We installed all of the server processes on one box, configured it to submit the quickstart indexing task to our hadoop cluster and it worked great.

The server processes are started using this script: https://github.com/druid-io/druid/blob/0.8.2/examples/bin/run_druid_server.sh

We figured it would be pretty straight forward to then leave all the configs the same, stop all the processes, create a similar script and then kick off a similar job from the CLI with some small changes to the .json index file. Unfortunately that isn’t the case, but it’s possible I’m doing it wrong.

I’ve had to repeat a lot things already set in the configs, some of which are documented here: http://druid.io/docs/latest/ingestion/command-line-hadoop-indexer.html

It’s important to note that all of the .xml files needed to get druid to see the hadoop cluster are just as they were when it ran successfully with the server processes (in _common/)

I’m specifying the 2.7.3 hadoop extensions to use in several places, but it doesn’t seem to be respecting them from the CLI, but it worked fine in the server processes. It is loading them, but not using them. I can see from the output that it’s not respecting the settings to use the 2.7.3 version of hadoop and it’s not submitting the job to the cluster at all and is instead attempting to run it locally.

Any help would be greatly appreciated.

Here is the script used to kick it off: (run_druid_indexer.sh)

#!/usr/bin/env bash

set +e

set +u

shopt -s xpg_echo

shopt -s expand_aliases

trap “exit 1” 1 2 3 15

SCRIPT_DIR=dirname $0

MAVEN_DIR="${SCRIPT_DIR}/extensions-repo"

SERVER_TYPE=“middleManager”

INDEX_JSON=$1

if [ “x${INDEX_JSON}” = “x” ]

then

echo “usage: $0 index_json” >& 2

exit 2

fi

if [[ ! -d “{SCRIPT_DIR}/lib" || ! -d "{SCRIPT_DIR}/config” ]]; then

echo “This script appears to be running from the source location. It must be run from its deployed location.”

echo “After building, unpack services/target/druid-services-*-SNAPSHOT-bin.tar.gz, and run the script unpacked there.”

exit 2

fi

CURR_DIR=pwd

cd ${SCRIPT_DIR}

SCRIPT_DIR=pwd

cd ${CURR_DIR}

start process

JAVA_ARGS="${JAVA_ARGS} -Xmx512m -Duser.timezone=UTC -Dfile.encoding=UTF-8"

JAVA_ARGS="{JAVA_ARGS} -Ddruid.extensions.localRepository={MAVEN_DIR}"

JAVA_ARGS="${JAVA_ARGS} -Ddruid.extensions.coordinates=[“io.druid.extensions:druid-hdfs-storage”,“org.apache.hadoop:hadoop-client:2.7.3”]"

JAVA_ARGS="${JAVA_ARGS} -Ddruid.indexer.task.defaultHadoopCoordinates=[“org.apache.hadoop:hadoop-client:2.7.3”,“org.apache.hadoop:hadoop-hdfs:2.7.3”]"

JAVA_ARGS="${JAVA_ARGS} -Ddruid.monitoring.monitors=[“com.metamx.metrics.JvmMonitor”]"

DRUID_CP="${SCRIPT_DIR}/config/_common/*"

DRUID_CP="{DRUID_CP}:{SCRIPT_DIR}/config/middleManager/*"

DRUID_CP="{DRUID_CP}:{SCRIPT_DIR}/config/coordinator/*"

DRUID_CP="{DRUID_CP}:{SCRIPT_DIR}/config/overlord/*"

DRUID_CP="{DRUID_CP}:{SCRIPT_DIR}/lib/*"

echo “JAVA_ARGS=${JAVA_ARGS}”

echo “DRUID_CP=${DRUID_CP}”

exec java {JAVA_ARGS} -classpath "{DRUID_CP}" io.druid.cli.Main index hadoop “$INDEX_JSON” -c io.druid.extensions:druid-hdfs-storage -c org.apache.hadoop:hadoop-client:2.7.3 -c org.apache.hadoop:hadoop-hdfs:2.7.3

Here is the .json configuration for the index task:

{

“type” : “index_hadoop”,

“spec” : {

“ioConfig” : {

“type” : “hadoop”,

“inputSpec” : {

“type” : “static”,

“paths” : “hdfs://processingHDFS2/tmp/wikiticker-2015-09-12-sampled.json”

},

“metadataUpdateSpec” : {

“type” :“derby”

},

“segmentOutputPath” : “/druid/segments”

},

“dataSchema” : {

“dataSource” : “wikiticker”,

“granularitySpec” : {

“type” : “uniform”,

“segmentGranularity” : “day”,

“queryGranularity” : “none”,

“intervals” : [“2015-09-12/2015-09-13”]

},

“parser” : {

“type” : “hadoopyString”,

“parseSpec” : {

“format” : “json”,

“dimensionsSpec” : {

“dimensions” : [

“channel”,

“cityName”,

“comment”,

“countryIsoCode”,

“countryName”,

“isAnonymous”,

“isMinor”,

“isNew”,

“isRobot”,

“isUnpatrolled”,

“metroCode”,

“namespace”,

“page”,

“regionIsoCode”,

“regionName”,

“user”

]

},

“timestampSpec” : {

“format” : “auto”,

“column” : “time”

}

}

},

“metricsSpec” : [

{

“name” : “count”,

“type” : “count”

},

{

“name” : “added”,

“type” : “longSum”,

“fieldName” : “added”

},

{

“name” : “deleted”,

“type” : “longSum”,

“fieldName” : “deleted”

},

{

“name” : “delta”,

“type” : “longSum”,

“fieldName” : “delta”

},

{

“name” : “user_unique”,

“type” : “hyperUnique”,

“fieldName” : “user”

}

]

},

“tuningConfig” : {

“type” : “hadoop”,

“workingPath” : “var/druid/hadoop-tmp”,

“partitionsSpec” : {

“type” : “hashed”,

“targetPartitionSize” : 5000000

},

“jobProperties” : {

“mapreduce.job.classloader”: “true”,

“mapreduce.job.classloader.system.classes”: “-javax.validation.,java.,javax.,org.apache.commons.logging.,org.apache.log4j.,org.apache.hadoop.,org.xerial.snappy.”

}

}

},

“hadoopDependencyCoordinates”: [

“org.apache.hadoop:hadoop-client:2.7.3”,

“org.apache.hadoop:hadoop-hdfs:2.7.3”

]

}

Here is the output:

./run_druid_indexer.sh /home/asher/druid-0.9.0/quickstart/wikiticker-index2.json

JAVA_ARGS= -Xmx512m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Ddruid.extensions.localRepository=./extensions-repo -Ddruid.extensions.coordinates=[“io.druid.extensions:druid-hdfs-storage”,“org.apache.hadoop:hadoop-client:2.7.3”] -Ddruid.indexer.task.defaultHadoopCoordinates=[“org.apache.hadoop:hadoop-client:2.7.3”,“org.apache.hadoop:hadoop-hdfs:2.7.3”] -Ddruid.monitoring.monitors=[“com.metamx.metrics.JvmMonitor”]

DRUID_CP=/opt/druid/config/_common/:/opt/druid/config/middleManager/:/opt/druid/config/coordinator/:/opt/druid/config/overlord/:/opt/druid/lib/*

Feb 01, 2017 5:27:43 PM org.hibernate.validator.internal.util.Version

INFO: HV000001: Hibernate Validator 5.1.3.Final

2017-02-01T17:27:44,291 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[io.druid.extensions:druid-hdfs-storage, org.apache.hadoop:hadoop-client:2.7.3], defaultVersion=‘0.8.2’, localRepository=’./extensions-repo’, remoteRepositories=[https://repo1.maven.org/maven2/, https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local]}]

2017-02-01T17:27:44,400 INFO [main] io.druid.initialization.Initialization - Loading extension[io.druid.extensions:druid-hdfs-storage] for class[io.druid.cli.CliCommandCreator]

2017-02-01T17:27:44,864 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/io/druid/extensions/druid-hdfs-storage/0.8.2/druid-hdfs-storage-0.8.2.jar]

2017-02-01T17:27:44,864 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/net/java/dev/jets3t/jets3t/0.9.4/jets3t-0.9.4.jar]

2017-02-01T17:27:44,864 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar]

2017-02-01T17:27:44,864 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar]

2017-02-01T17:27:44,864 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/activation/activation/1.1.1/activation-1.1.1.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/bouncycastle/bcprov-jdk15on/1.52/bcprov-jdk15on-1.52.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/jamesmurty/utils/java-xmlbuilder/1.1/java-xmlbuilder-1.1.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/net/iharder/base64/2.3.8/base64-2.3.8.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/httpcomponents/httpclient/4.5.1/httpclient-4.5.1.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/httpcomponents/httpcore/4.4.3/httpcore-4.4.3.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-client/2.3.0/hadoop-client-2.3.0.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-common/2.3.0/hadoop-common-2.3.0.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-cli/commons-cli/1.2/commons-cli-1.2.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xmlenc/xmlenc/0.52/xmlenc-0.52.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar]

2017-02-01T17:27:44,865 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-net/commons-net/3.1/commons-net-3.1.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/log4j/log4j/1.2.17/log4j-1.2.17.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-lang/commons-lang/2.6/commons-lang-2.6.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-digester/commons-digester/1.8/commons-digester-1.8.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/avro/avro/1.7.4/avro-1.7.4.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/xerial/snappy/snappy-java/1.0.4.1/snappy-java-1.0.4.1.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar]

2017-02-01T17:27:44,866 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-auth/2.3.0/hadoop-auth-2.3.0.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/tukaani/xz/1.0/xz-1.0.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-hdfs/2.3.0/hadoop-hdfs-2.3.0.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-app/2.3.0/hadoop-mapreduce-client-app-2.3.0.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-common/2.3.0/hadoop-mapreduce-client-common-2.3.0.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-client/2.3.0/hadoop-yarn-client-2.3.0.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-server-common/2.3.0/hadoop-yarn-server-common-2.3.0.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-shuffle/2.3.0/hadoop-mapreduce-client-shuffle-2.3.0.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-api/2.3.0/hadoop-yarn-api-2.3.0.jar]

2017-02-01T17:27:44,867 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-core/2.3.0/hadoop-mapreduce-client-core-2.3.0.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-common/2.3.0/hadoop-yarn-common-2.3.0.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/sun/jersey/jersey-core/1.19/jersey-core-1.19.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.3.0/hadoop-mapreduce-client-jobclient-2.3.0.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-annotations/2.3.0/hadoop-annotations-2.3.0.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/metamx/emitter/0.3.2/emitter-0.3.2.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/metamx/http-client/1.0.4/http-client-1.0.4.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/io/netty/netty/3.10.4.Final/netty-3.10.4.Final.jar]

2017-02-01T17:27:44,868 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-io/commons-io/2.0.1/commons-io-2.0.1.jar]

2017-02-01T17:27:44,902 INFO [main] io.druid.initialization.Initialization - Loading extension[org.apache.hadoop:hadoop-client:2.7.3] for class[io.druid.cli.CliCommandCreator]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-common/2.7.3/hadoop-common-2.7.3.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/guava/guava/11.0.2/guava-11.0.2.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-cli/commons-cli/1.2/commons-cli-1.2.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xmlenc/xmlenc/0.52/xmlenc-0.52.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-codec/commons-codec/1.4/commons-codec-1.4.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-io/commons-io/2.4/commons-io-2.4.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-net/commons-net/3.1/commons-net-3.1.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar]

2017-02-01T17:27:45,057 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/log4j/log4j/1.2.17/log4j-1.2.17.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-lang/commons-lang/2.6/commons-lang-2.6.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-digester/commons-digester/1.8/commons-digester-1.8.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/slf4j/slf4j-api/1.7.10/slf4j-api-1.7.10.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/avro/avro/1.7.4/avro-1.7.4.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar]

2017-02-01T17:27:45,058 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/xerial/snappy/snappy-java/1.0.4.1/snappy-java-1.0.4.1.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-auth/2.7.3/hadoop-auth-2.7.3.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/httpcomponents/httpclient/4.2.5/httpclient-4.2.5.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/httpcomponents/httpcore/4.2.5/httpcore-4.2.5.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/directory/server/apacheds-kerberos-codec/2.0.0-M15/apacheds-kerberos-codec-2.0.0-M15.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/directory/server/apacheds-i18n/2.0.0-M15/apacheds-i18n-2.0.0-M15.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/directory/api/api-asn1-api/1.0.0-M20/api-asn1-api-1.0.0-M20.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/directory/api/api-util/1.0.0-M20/api-util-1.0.0-M20.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/curator/curator-framework/2.7.1/curator-framework-2.7.1.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/curator/curator-client/2.7.1/curator-client-2.7.1.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/curator/curator-recipes/2.7.1/curator-recipes-2.7.1.jar]

2017-02-01T17:27:45,059 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/code/findbugs/jsr305/3.0.0/jsr305-3.0.0.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/tukaani/xz/1.0/xz-1.0.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-hdfs/2.7.3/hadoop-hdfs-2.7.3.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/io/netty/netty/3.6.2.Final/netty-3.6.2.Final.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/io/netty/netty-all/4.0.23.Final/netty-all-4.0.23.Final.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xerces/xercesImpl/2.9.1/xercesImpl-2.9.1.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xml-apis/xml-apis/1.3.04/xml-apis-1.3.04.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-app/2.7.3/hadoop-mapreduce-client-app-2.7.3.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-common/2.7.3/hadoop-mapreduce-client-common-2.7.3.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-client/2.7.3/hadoop-yarn-client-2.7.3.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-server-common/2.7.3/hadoop-yarn-server-common-2.7.3.jar]

2017-02-01T17:27:45,060 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-shuffle/2.7.3/hadoop-mapreduce-client-shuffle-2.7.3.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-api/2.7.3/hadoop-yarn-api-2.7.3.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-core/2.7.3/hadoop-mapreduce-client-core-2.7.3.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-common/2.7.3/hadoop-yarn-common-2.7.3.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/activation/activation/1.1/activation-1.1.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/sun/jersey/jersey-core/1.9/jersey-core-1.9.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/sun/jersey/jersey-client/1.9/jersey-client-1.9.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-jaxrs/1.9.13/jackson-jaxrs-1.9.13.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-xc/1.9.13/jackson-xc-1.9.13.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.7.3/hadoop-mapreduce-client-jobclient-2.7.3.jar]

2017-02-01T17:27:45,061 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-annotations/2.7.3/hadoop-annotations-2.7.3.jar]

2017-02-01T17:27:45,176 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-hdfs/2.7.3/hadoop-hdfs-2.7.3.jar]

2017-02-01T17:27:45,176 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/guava/guava/11.0.2/guava-11.0.2.jar]

2017-02-01T17:27:45,176 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/code/findbugs/jsr305/3.0.0/jsr305-3.0.0.jar]

2017-02-01T17:27:45,176 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar]

2017-02-01T17:27:45,176 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar]

2017-02-01T17:27:45,176 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/sun/jersey/jersey-core/1.9/jersey-core-1.9.jar]

2017-02-01T17:27:45,176 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/sun/jersey/jersey-server/1.9/jersey-server-1.9.jar]

2017-02-01T17:27:45,176 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/asm/asm/3.2/asm-3.2.jar]

2017-02-01T17:27:45,176 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-cli/commons-cli/1.2/commons-cli-1.2.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-codec/commons-codec/1.4/commons-codec-1.4.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-io/commons-io/2.4/commons-io-2.4.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-lang/commons-lang/2.6/commons-lang-2.6.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-daemon/commons-daemon/1.0.13/commons-daemon-1.0.13.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/log4j/log4j/1.2.17/log4j-1.2.17.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xmlenc/xmlenc/0.52/xmlenc-0.52.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/io/netty/netty/3.6.2.Final/netty-3.6.2.Final.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/io/netty/netty-all/4.0.23.Final/netty-all-4.0.23.Final.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xerces/xercesImpl/2.9.1/xercesImpl-2.9.1.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xml-apis/xml-apis/1.3.04/xml-apis-1.3.04.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar]

2017-02-01T17:27:45,177 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-client/2.3.0/hadoop-client-2.3.0.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-common/2.3.0/hadoop-common-2.3.0.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/guava/guava/11.0.2/guava-11.0.2.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-cli/commons-cli/1.2/commons-cli-1.2.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xmlenc/xmlenc/0.52/xmlenc-0.52.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-codec/commons-codec/1.4/commons-codec-1.4.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-io/commons-io/2.4/commons-io-2.4.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-net/commons-net/3.1/commons-net-3.1.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar]

2017-02-01T17:27:45,286 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/log4j/log4j/1.2.17/log4j-1.2.17.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-lang/commons-lang/2.6/commons-lang-2.6.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-digester/commons-digester/1.8/commons-digester-1.8.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/avro/avro/1.7.4/avro-1.7.4.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/xerial/snappy/snappy-java/1.0.4.1/snappy-java-1.0.4.1.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-auth/2.3.0/hadoop-auth-2.3.0.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/httpcomponents/httpclient/4.2.5/httpclient-4.2.5.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/httpcomponents/httpcore/4.2.5/httpcore-4.2.5.jar]

2017-02-01T17:27:45,287 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/tukaani/xz/1.0/xz-1.0.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-hdfs/2.3.0/hadoop-hdfs-2.3.0.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-app/2.3.0/hadoop-mapreduce-client-app-2.3.0.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-common/2.3.0/hadoop-mapreduce-client-common-2.3.0.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-client/2.3.0/hadoop-yarn-client-2.3.0.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-server-common/2.3.0/hadoop-yarn-server-common-2.3.0.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-shuffle/2.3.0/hadoop-mapreduce-client-shuffle-2.3.0.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-api/2.3.0/hadoop-yarn-api-2.3.0.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-core/2.3.0/hadoop-mapreduce-client-core-2.3.0.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-common/2.3.0/hadoop-yarn-common-2.3.0.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar]

2017-02-01T17:27:45,288 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar]

2017-02-01T17:27:45,289 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/activation/activation/1.1/activation-1.1.jar]

2017-02-01T17:27:45,289 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar]

2017-02-01T17:27:45,289 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/sun/jersey/jersey-core/1.9/jersey-core-1.9.jar]

2017-02-01T17:27:45,289 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.3.0/hadoop-mapreduce-client-jobclient-2.3.0.jar]

2017-02-01T17:27:45,289 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-annotations/2.3.0/hadoop-annotations-2.3.0.jar]

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/opt/druid-0.8.2/lib/log4j-slf4j-impl-2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/opt/druid-0.8.2/./extensions-repo/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/opt/druid-0.8.2/./extensions-repo/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

2017-02-01T17:27:45,756 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.1.3.Final

2017-02-01T17:27:46,142 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[io.druid.extensions:druid-hdfs-storage, org.apache.hadoop:hadoop-client:2.7.3], defaultVersion=‘0.8.2’, localRepository=’./extensions-repo’, remoteRepositories=[https://repo1.maven.org/maven2/, https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local]}]

2017-02-01T17:27:46,252 INFO [main] io.druid.initialization.Initialization - Loading extension[io.druid.extensions:druid-hdfs-storage] for class[io.druid.cli.CliCommandCreator]

2017-02-01T17:27:46,683 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/io/druid/extensions/druid-hdfs-storage/0.8.2/druid-hdfs-storage-0.8.2.jar]

2017-02-01T17:27:46,683 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/net/java/dev/jets3t/jets3t/0.9.4/jets3t-0.9.4.jar]

2017-02-01T17:27:46,683 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar]

2017-02-01T17:27:46,683 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar]

2017-02-01T17:27:46,683 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/activation/activation/1.1.1/activation-1.1.1.jar]

2017-02-01T17:27:46,683 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar]

2017-02-01T17:27:46,683 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/bouncycastle/bcprov-jdk15on/1.52/bcprov-jdk15on-1.52.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/jamesmurty/utils/java-xmlbuilder/1.1/java-xmlbuilder-1.1.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/net/iharder/base64/2.3.8/base64-2.3.8.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/httpcomponents/httpclient/4.5.1/httpclient-4.5.1.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/httpcomponents/httpcore/4.4.3/httpcore-4.4.3.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-client/2.3.0/hadoop-client-2.3.0.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-common/2.3.0/hadoop-common-2.3.0.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-cli/commons-cli/1.2/commons-cli-1.2.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xmlenc/xmlenc/0.52/xmlenc-0.52.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-net/commons-net/3.1/commons-net-3.1.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/log4j/log4j/1.2.17/log4j-1.2.17.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-lang/commons-lang/2.6/commons-lang-2.6.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-digester/commons-digester/1.8/commons-digester-1.8.jar]

2017-02-01T17:27:46,684 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/avro/avro/1.7.4/avro-1.7.4.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/xerial/snappy/snappy-java/1.0.4.1/snappy-java-1.0.4.1.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-auth/2.3.0/hadoop-auth-2.3.0.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/tukaani/xz/1.0/xz-1.0.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-hdfs/2.3.0/hadoop-hdfs-2.3.0.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-app/2.3.0/hadoop-mapreduce-client-app-2.3.0.jar]

2017-02-01T17:27:46,685 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-common/2.3.0/hadoop-mapreduce-client-common-2.3.0.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-client/2.3.0/hadoop-yarn-client-2.3.0.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-server-common/2.3.0/hadoop-yarn-server-common-2.3.0.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-shuffle/2.3.0/hadoop-mapreduce-client-shuffle-2.3.0.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-api/2.3.0/hadoop-yarn-api-2.3.0.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-core/2.3.0/hadoop-mapreduce-client-core-2.3.0.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-common/2.3.0/hadoop-yarn-common-2.3.0.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/sun/jersey/jersey-core/1.19/jersey-core-1.19.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.3.0/hadoop-mapreduce-client-jobclient-2.3.0.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-annotations/2.3.0/hadoop-annotations-2.3.0.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/metamx/emitter/0.3.2/emitter-0.3.2.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/metamx/http-client/1.0.4/http-client-1.0.4.jar]

2017-02-01T17:27:46,686 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/io/netty/netty/3.10.4.Final/netty-3.10.4.Final.jar]

2017-02-01T17:27:46,687 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-io/commons-io/2.0.1/commons-io-2.0.1.jar]

2017-02-01T17:27:46,695 INFO [main] io.druid.initialization.Initialization - Loading extension[org.apache.hadoop:hadoop-client:2.7.3] for class[io.druid.cli.CliCommandCreator]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-common/2.7.3/hadoop-common-2.7.3.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/guava/guava/11.0.2/guava-11.0.2.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-cli/commons-cli/1.2/commons-cli-1.2.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xmlenc/xmlenc/0.52/xmlenc-0.52.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-codec/commons-codec/1.4/commons-codec-1.4.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-io/commons-io/2.4/commons-io-2.4.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-net/commons-net/3.1/commons-net-3.1.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/log4j/log4j/1.2.17/log4j-1.2.17.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-lang/commons-lang/2.6/commons-lang-2.6.jar]

2017-02-01T17:27:46,842 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-digester/commons-digester/1.8/commons-digester-1.8.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/slf4j/slf4j-api/1.7.10/slf4j-api-1.7.10.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/avro/avro/1.7.4/avro-1.7.4.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/xerial/snappy/snappy-java/1.0.4.1/snappy-java-1.0.4.1.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-auth/2.7.3/hadoop-auth-2.7.3.jar]

2017-02-01T17:27:46,843 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/httpcomponents/httpclient/4.2.5/httpclient-4.2.5.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/httpcomponents/httpcore/4.2.5/httpcore-4.2.5.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/directory/server/apacheds-kerberos-codec/2.0.0-M15/apacheds-kerberos-codec-2.0.0-M15.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/directory/server/apacheds-i18n/2.0.0-M15/apacheds-i18n-2.0.0-M15.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/directory/api/api-asn1-api/1.0.0-M20/api-asn1-api-1.0.0-M20.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/directory/api/api-util/1.0.0-M20/api-util-1.0.0-M20.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/curator/curator-framework/2.7.1/curator-framework-2.7.1.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/curator/curator-client/2.7.1/curator-client-2.7.1.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/curator/curator-recipes/2.7.1/curator-recipes-2.7.1.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/google/code/findbugs/jsr305/3.0.0/jsr305-3.0.0.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/tukaani/xz/1.0/xz-1.0.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-hdfs/2.7.3/hadoop-hdfs-2.7.3.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/io/netty/netty/3.6.2.Final/netty-3.6.2.Final.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/io/netty/netty-all/4.0.23.Final/netty-all-4.0.23.Final.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xerces/xercesImpl/2.9.1/xercesImpl-2.9.1.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/xml-apis/xml-apis/1.3.04/xml-apis-1.3.04.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-app/2.7.3/hadoop-mapreduce-client-app-2.7.3.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-common/2.7.3/hadoop-mapreduce-client-common-2.7.3.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-client/2.7.3/hadoop-yarn-client-2.7.3.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-server-common/2.7.3/hadoop-yarn-server-common-2.7.3.jar]

2017-02-01T17:27:46,844 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-shuffle/2.7.3/hadoop-mapreduce-client-shuffle-2.7.3.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-api/2.7.3/hadoop-yarn-api-2.7.3.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-core/2.7.3/hadoop-mapreduce-client-core-2.7.3.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-yarn-common/2.7.3/hadoop-yarn-common-2.7.3.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/activation/activation/1.1/activation-1.1.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/sun/jersey/jersey-core/1.9/jersey-core-1.9.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/com/sun/jersey/jersey-client/1.9/jersey-client-1.9.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-jaxrs/1.9.13/jackson-jaxrs-1.9.13.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/codehaus/jackson/jackson-xc/1.9.13/jackson-xc-1.9.13.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.7.3/hadoop-mapreduce-client-jobclient-2.7.3.jar]

2017-02-01T17:27:46,845 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.2/./extensions-repo/org/apache/hadoop/hadoop-annotations/2.7.3/hadoop-annotations-2.7.3.jar]

2017-02-01T17:27:47,023 INFO [main] io.druid.initialization.Initialization - Loading extension[io.druid.extensions:druid-hdfs-storage] for class[io.druid.initialization.DruidModule]

2017-02-01T17:27:47,025 INFO [main] io.druid.initialization.Initialization - Adding remote extension module[io.druid.storage.hdfs.HdfsStorageDruidModule] for class[io.druid.initialization.DruidModule]

2017-02-01T17:27:47,025 INFO [main] io.druid.initialization.Initialization - Loading extension[org.apache.hadoop:hadoop-client:2.7.3] for class[io.druid.initialization.DruidModule]

2017-02-01T17:27:47,307 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[io.druid.extensions:druid-hdfs-storage, org.apache.hadoop:hadoop-client:2.7.3], defaultVersion=‘0.8.2’, localRepository=’./extensions-repo’, remoteRepositories=[https://repo1.maven.org/maven2/, https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local]}]

2017-02-01T17:27:47,309 INFO [main] io.druid.initialization.Initialization - Loading extension[io.druid.extensions:druid-hdfs-storage] for class[io.druid.initialization.DruidModule]

2017-02-01T17:27:47,310 INFO [main] io.druid.initialization.Initialization - Adding remote extension module[io.druid.storage.hdfs.HdfsStorageDruidModule] for class[io.druid.initialization.DruidModule]

2017-02-01T17:27:47,310 INFO [main] io.druid.initialization.Initialization - Loading extension[org.apache.hadoop:hadoop-client:2.7.3] for class[io.druid.initialization.DruidModule]

2017-02-01T17:27:47,477 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

2017-02-01T17:27:47,771 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [io.druid.server.metrics.DruidMonitorSchedulerConfig@71fb8301]

2017-02-01T17:27:47,779 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[class com.metamx.metrics.JvmMonitor]}]

2017-02-01T17:27:47,783 INFO [main] io.druid.server.metrics.MetricsModule - Adding monitor[com.metamx.metrics.JvmMonitor@18dac12f]

2017-02-01T17:27:47,899 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.computation.buffer.size, ${base_path}.buffer.sizeBytes] on [io.druid.query.DruidProcessingConfig#intermediateComputeSizeBytes()]

2017-02-01T17:27:47,900 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.numThreads] on [io.druid.query.DruidProcessingConfig#getNumThreads()]

2017-02-01T17:27:47,900 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.columnCache.sizeBytes] on [io.druid.query.DruidProcessingConfig#columnCacheSizeBytes()]

2017-02-01T17:27:47,900 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning default value [processing-%s] for [${base_path}.formatString] on [com.metamx.common.concurrent.ExecutorServiceConfig#getFormatString()]

2017-02-01T17:27:47,958 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.segment.data.BitmapSerdeFactory] from props[druid.processing.bitmap.] as [ConciseBitmapSerdeFactory{}]

Feb 01, 2017 5:27:47 PM com.google.inject.servlet.GuiceFilter setPipeline

WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected.

2017-02-01T17:27:48,004 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [io.druid.server.metrics.DruidMonitorSchedulerConfig@5256fbc]

2017-02-01T17:27:48,007 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[class com.metamx.metrics.JvmMonitor]}]

2017-02-01T17:27:48,012 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.DruidNode] from props[druid.] as [DruidNode{serviceName=‘druid/internal-hadoop-indexer’, host=‘localhost’, port=0}]

2017-02-01T17:27:48,012 INFO [main] io.druid.server.metrics.MetricsModule - Adding monitor[com.metamx.metrics.JvmMonitor@248b2b61]

2017-02-01T17:27:48,017 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.metadata.MetadataStorageTablesConfig] from props[druid.metadata.storage.tables.] as [io.druid.metadata.MetadataStorageTablesConfig@3fe98084]

2017-02-01T17:27:48,087 INFO [main] io.druid.metadata.storage.derby.DerbyConnector - Configured Derby as metadata storage

2017-02-01T17:27:48,129 INFO [main] io.druid.indexer.HadoopDruidIndexerConfig - Running with config:

{

“spec” : {

“dataSchema” : {

“dataSource” : “wikiticker”,

“parser” : {

“type” : “hadoopyString”,

“parseSpec” : {

“format” : “json”,

“dimensionsSpec” : {

“dimensions” : [ “channel”, “cityName”, “comment”, “countryIsoCode”, “countryName”, “isAnonymous”, “isMinor”, “isNew”, “isRobot”, “isUnpatrolled”, “metroCode”, “namespace”, “page”, “regionIsoCode”, “regionName”, “user” ]

},

“timestampSpec” : {

“format” : “auto”,

“column” : “time”

}

}

},

“metricsSpec” : [ {

“type” : “count”,

“name” : “count”

}, {

“type” : “longSum”,

“name” : “added”,

“fieldName” : “added”

}, {

“type” : “longSum”,

“name” : “deleted”,

“fieldName” : “deleted”

}, {

“type” : “longSum”,

“name” : “delta”,

“fieldName” : “delta”

}, {

“type” : “hyperUnique”,

“name” : “user_unique”,

“fieldName” : “user”

} ],

“granularitySpec” : {

“type” : “uniform”,

“segmentGranularity” : “DAY”,

“queryGranularity” : {

“type” : “none”

},

“intervals” : [ “2015-09-12T00:00:00.000Z/2015-09-13T00:00:00.000Z” ]

}

},

“ioConfig” : {

“type” : “hadoop”,

“inputSpec” : {

“type” : “static”,

“paths” : “hdfs://processingHDFS2/tmp/wikiticker-2015-09-12-sampled.json”

},

“metadataUpdateSpec” : {

“type” : “derby”,

“connectURI” : null,

“user” : null,

“password” : null,

“segmentTable” : null

},

“segmentOutputPath” : “/druid/segments”

},

“tuningConfig” : {

“type” : “hadoop”,

“workingPath” : “var/druid/hadoop-tmp”,

“version” : “2017-02-01T17:27:47.958Z”,

“partitionsSpec” : {

“type” : “hashed”,

“targetPartitionSize” : 5000000,

“maxPartitionSize” : 7500000,

“assumeGrouped” : false,

“numShards” : -1

},

“shardSpecs” : { },

“indexSpec” : {

“bitmap” : {

“type” : “concise”

},

“dimensionCompression” : null,

“metricCompression” : null

},

“leaveIntermediate” : false,

“cleanupOnFailure” : true,

“overwriteFiles” : false,

“ignoreInvalidRows” : false,

“jobProperties” : {

“mapreduce.job.classloader” : “true”,

“mapreduce.job.classloader.system.classes” : “-javax.validation.,java.,javax.,org.apache.commons.logging.,org.apache.log4j.,org.apache.hadoop.,org.xerial.snappy.”

},

“combineText” : false,

“persistInHeap” : false,

“ingestOffheap” : false,

“bufferSize” : 134217728,

“aggregationBufferRatio” : 0.5,

“useCombiner” : false,

“rowFlushBoundary” : 80000

}

}

}

2017-02-01T17:27:48,177 INFO [main] io.druid.indexer.path.StaticPathSpec - Adding paths[hdfs://processingHDFS2/tmp/wikiticker-2015-09-12-sampled.json]

2017-02-01T17:27:48,198 INFO [main] io.druid.indexer.path.StaticPathSpec - Adding paths[hdfs://processingHDFS2/tmp/wikiticker-2015-09-12-sampled.json]

2017-02-01T17:27:48,209 INFO [main] org.apache.hadoop.conf.Configuration.deprecation - session.id is deprecated. Instead, use dfs.metrics.session-id

2017-02-01T17:27:48,209 INFO [main] org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId=

2017-02-01T17:27:48,262 WARN [main] org.apache.hadoop.mapreduce.JobSubmitter - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.

2017-02-01T17:27:48,265 WARN [main] org.apache.hadoop.mapreduce.JobSubmitter - No job jar file set. User classes may not be found. See Job or Job#setJar(String).

2017-02-01T17:27:48,318 INFO [main] org.apache.hadoop.mapreduce.JobSubmitter - Cleaning up the staging area file:/tmp/hadoop-asher/mapred/staging/asher882329375/.staging/job_local882329375_0001

2017-02-01T17:27:48,320 ERROR [main] io.druid.cli.CliHadoopIndexer - failure!!!

java.lang.reflect.InvocationTargetException

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]

at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]

at io.druid.cli.CliHadoopIndexer.run(CliHadoopIndexer.java:120) [druid-services-0.8.2.jar:0.8.2]

at io.druid.cli.Main.main(Main.java:91) [druid-services-0.8.2.jar:0.8.2]

Caused by: java.lang.IllegalArgumentException: java.net.UnknownHostException: processingHDFS2

at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:237) ~[hadoop-hdfs-2.3.0.jar:?]

at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:141) ~[hadoop-hdfs-2.3.0.jar:?]

at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:569) ~[hadoop-hdfs-2.3.0.jar:?]

at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:512) ~[hadoop-hdfs-2.3.0.jar:?]

at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:142) ~[hadoop-hdfs-2.3.0.jar:?]

at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2316) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:90) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2350) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2332) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:369) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:446) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat.getSplits(DelegatingInputFormat.java:110) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_121]

at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_121]

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at io.druid.indexer.DetermineHashedPartitionsJob.run(DetermineHashedPartitionsJob.java:111) ~[druid-indexing-hadoop-0.8.2.jar:0.8.2]

at io.druid.indexer.JobHelper.runJobs(JobHelper.java:182) ~[druid-indexing-hadoop-0.8.2.jar:0.8.2]

at io.druid.indexer.HadoopDruidDetermineConfigurationJob.run(HadoopDruidDetermineConfigurationJob.java:84) ~[druid-indexing-hadoop-0.8.2.jar:0.8.2]

at io.druid.indexer.JobHelper.runJobs(JobHelper.java:182) ~[druid-indexing-hadoop-0.8.2.jar:0.8.2]

at io.druid.cli.CliInternalHadoopIndexer.run(CliInternalHadoopIndexer.java:132) ~[druid-services-0.8.2.jar:0.8.2]

at io.druid.cli.Main.main(Main.java:91) ~[druid-services-0.8.2.jar:0.8.2]

… 6 more

Caused by: java.net.UnknownHostException: processingHDFS2

at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:237) ~[hadoop-hdfs-2.3.0.jar:?]

at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:141) ~[hadoop-hdfs-2.3.0.jar:?]

at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:569) ~[hadoop-hdfs-2.3.0.jar:?]

at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:512) ~[hadoop-hdfs-2.3.0.jar:?]

at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:142) ~[hadoop-hdfs-2.3.0.jar:?]

at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2316) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:90) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2350) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2332) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:369) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:446) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat.getSplits(DelegatingInputFormat.java:110) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_121]

at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_121]

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) ~[hadoop-common-2.3.0.jar:?]

at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at io.druid.indexer.DetermineHashedPartitionsJob.run(DetermineHashedPartitionsJob.java:111) ~[druid-indexing-hadoop-0.8.2.jar:0.8.2]

at io.druid.indexer.JobHelper.runJobs(JobHelper.java:182) ~[druid-indexing-hadoop-0.8.2.jar:0.8.2]

at io.druid.indexer.HadoopDruidDetermineConfigurationJob.run(HadoopDruidDetermineConfigurationJob.java:84) ~[druid-indexing-hadoop-0.8.2.jar:0.8.2]

at io.druid.indexer.JobHelper.runJobs(JobHelper.java:182) ~[druid-indexing-hadoop-0.8.2.jar:0.8.2]

at io.druid.cli.CliInternalHadoopIndexer.run(CliInternalHadoopIndexer.java:132) ~[druid-services-0.8.2.jar:0.8.2]

at io.druid.cli.Main.main(Main.java:91) ~[druid-services-0.8.2.jar:0.8.2]

… 6 more

Considering building a custom megajar and just changing the hadoop.compile.version, but it would be great if I didn’t have to do this. That doesn’t help with it ignoring the _common directory though. I would rather handle this through config. It seems like a couple changes in the right places could help here… any suggestions?

I was able to get this to work without a lot of fuss and it was a lot cleaner. Responding to myself for the good of the group. It just needs a VERY specific config.
If you have the proper xml files in your _common folder the command you need to issue looks like this:

java -Xmx512m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/_common io.druid.cli.Main index hadoop /tmp/quickstart/wikiticker-index2.json

and the config I am using looks like this:

{

“type” : “index_hadoop”,

“spec” : {

“ioConfig” : {

“type” : “hadoop”,

“inputSpec” : {

“type” : “static”,

“paths” : “hdfs://myhdfsnamespace/tmp/wikiticker-2015-09-12-sampled.json”

},

“metadataUpdateSpec” : {

“type” :“derby”

},

“segmentOutputPath” : “/druid/segments”

},

“dataSchema” : {

“dataSource” : “wikiticker”,

“granularitySpec” : {

“type” : “uniform”,

“segmentGranularity” : “day”,

“queryGranularity” : “none”,

“intervals” : [“2015-09-12/2015-09-13”]

},

“parser” : {

“type” : “hadoopyString”,

“parseSpec” : {

“format” : “json”,

“dimensionsSpec” : {

“dimensions” : [

“channel”,

“cityName”,

“comment”,

“countryIsoCode”,

“countryName”,

“isAnonymous”,

“isMinor”,

“isNew”,

“isRobot”,

“isUnpatrolled”,

“metroCode”,

“namespace”,

“page”,

“regionIsoCode”,

“regionName”,

“user”

]

},

“timestampSpec” : {

“format” : “auto”,

“column” : “time”

}

}

},

“metricsSpec” : [

{

“name” : “count”,

“type” : “count”

},

{

“name” : “added”,

“type” : “longSum”,

“fieldName” : “added”

},

{

“name” : “deleted”,

“type” : “longSum”,

“fieldName” : “deleted”

},

{

“name” : “delta”,

“type” : “longSum”,

“fieldName” : “delta”

},

{

“name” : “user_unique”,

“type” : “hyperUnique”,

“fieldName” : “user”

}

]

},

“tuningConfig” : {

“type” : “hadoop”,

“workingPath” : “var/druid/hadoop-tmp”,

“partitionsSpec” : {

“type” : “hashed”,

“targetPartitionSize” : 5000000

},

“jobProperties” : {

}

}

},

“hadoopDependencyCoordinates”: [

“org.apache.hadoop:hadoop-client:2.7.3”,

“org.apache.hadoop:hadoop-hdfs:2.7.3”

]

}