Index task from static-S3 firehose not working (0.10)

Hello,

I am new to druid and I have the following issue:

Version: druid 0.10.0

Deep storage in S3.

I ran an ingest command like:

$curl -X ‘POST’ -H ‘Content-Type:application/json’ -d @~/testSampleS3_spec.json X.X.X.X:8090/druid/indexer/v1/task

I receive the following:

{“error”:“Could not resolve type id ‘static-s3’ into a subtype of [simple type, class io.druid.data.input.FirehoseFactory]\n at [Source: HttpInputOverHTTP@664dc91a[c=951,q=1,[0]=EOF,s=STREAM]; line: 1, column: 808]”}

The contents of testSampleS3_spec.json:

{

“type”:“index”,

“spec”:{

“dataSchema” : {

“dataSource” : “sampleTableS3”,

“parser”: {

“type” : “string”,

“parseSpec”: {

“format”:“csv”,

“timestampSpec” : {

“column” : “timeTS”,

“format” : “auto”

},

“dimensionsSpec” : {

“dimensions”:[{“type”:“long”,“name”:“A”}, “B”],

“dimensionExclusions” : ,

“spatialDimensions” :

},

“columns” : [“timeTS”,“A”,“B”,“C”,“D”]

}

},

“metricsSpec” : [{“type” : “count”,“name” : “C”},{“type” : “count”,“name” : “D”}],

“granularitySpec” : {

“segmentGranularity” : “MONTH”,

“queryGranularity” : “NONE”,

“intervals” : [ “2016-01-01/2016-04-30” ]

}

},

“ioConfig”: {

“type” : “index”,

 "firehose" : {

   "type" : "static-s3",

   "uris" : ["s3://test/testSampleS3.txt"]

 }

},

“tuningConfig” : {

“type”: “index”

}

}

Here is the common.runtime.properties:

druid.extensions.loadList=[ “druid-s3-extensions”, “druid-histogram”, “druid-datasketches”, “druid-lookups-cached-global”, “mysql-metadata-storage”]

Here the top part of the overlord log:

2017-08-25T00:52:36,158 INFO [main] io.druid.guice.PropertiesModule - Loading properties from common.runtime.properties

2017-08-25T00:52:36,161 INFO [main] io.druid.guice.PropertiesModule - Loading properties from runtime.properties

2017-08-25T00:52:36,202 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.1.3.Final

2017-08-25T00:52:36,831 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClas

sloader=true, directory=’/druid/extensions’, hadoopDependenciesDir=‘hadoop-dependencies’, hadoopContainerDruidClasspath=‘null’, loadList=[druid-s3-extensions, druid-histogram, druid-datasket

ches, druid-lookups-cached-global, mysql-metadata-storage]}]

2017-08-25T00:52:36,834 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-s3-extensions] for class [io.druid.cli.CliCommandCreator]

2017-08-25T00:52:36,846 INFO [main] io.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-s3-extensions/druid-s3-extensions-0.10.0.jar]

2017-08-25T00:52:36,847 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-histogram] for class [io.druid.cli.CliCommandCreator]

2017-08-25T00:52:36,847 INFO [main] io.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-histogram/druid-histogram-0.10.0.jar]

2017-08-25T00:52:36,848 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-datasketches] for class [io.druid.cli.CliCommandCreator]

2017-08-25T00:52:36,848 INFO [main] io.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-datasketches/sketches-core-0.8.4.jar]

2017-08-25T00:52:36,849 INFO [main] io.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-datasketches/druid-datasketches-0.10.0.jar]

2017-08-25T00:52:36,849 INFO [main] io.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-datasketches/memory-0.8.4.jar]

2017-08-25T00:52:36,850 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-lookups-cached-global] for class [io.druid.cli.CliCommandCreator]

2017-08-25T00:52:36,850 INFO [main] io.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-lookups-cached-global/druid-lookups-cached-global-0.10.0.jar]

2017-08-25T00:52:36,851 INFO [main] io.druid.initialization.Initialization - Loading extension [mysql-metadata-storage] for class [io.druid.cli.CliCommandCreator]

2017-08-25T00:52:36,851 INFO [main] io.druid.initialization.Initialization - added URL[file:/druid/extensions/mysql-metadata-storage/mysql-connector-java-5.1.38.jar]

2017-08-25T00:52:36,851 INFO [main] io.druid.initialization.Initialization - added URL[file:/druid/extensions/mysql-metadata-storage/mysql-metadata-storage-0.10.1.jar]

2017-08-25T00:52:37,058 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-s3-extensions] for class [io.druid.initialization.DruidModule]

2017-08-25T00:52:37,227 INFO [main] io.druid.initialization.Initialization - Adding local file system extension module [io.druid.storage.s3.S3StorageDruidModule] for class [io.druid.initiali

zation.DruidModule]

2017-08-25T00:52:37,228 INFO [main] io.druid.initialization.Initialization - Adding local file system extension module [io.druid.firehose.s3.S3FirehoseDruidModule] for class [io.druid.initia

lization.DruidModule]

2017-08-25T00:52:37,228 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-histogram] for class [io.druid.initialization.DruidModule]

2017-08-25T00:52:37,231 INFO [main] io.druid.initialization.Initialization - Adding local file system extension module [io.druid.query.aggregation.histogram.ApproximateHistogramDruidModule]

for class [io.druid.initialization.DruidModule]

2017-08-25T00:52:37,231 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-datasketches] for class [io.druid.initialization.DruidModule]

2017-08-25T00:52:37,235 INFO [main] io.druid.initialization.Initialization - Adding local file system extension module [io.druid.query.aggregation.datasketches.theta.SketchModule] for class

[io.druid.initialization.DruidModule]

2017-08-25T00:52:37,243 INFO [main] io.druid.initialization.Initialization - Adding local file system extension module [io.druid.query.aggregation.datasketches.theta.oldapi.OldApiSketchModul

e] for class [io.druid.initialization.DruidModule]

2017-08-25T00:52:37,243 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-lookups-cached-global] for class [io.druid.initialization.DruidModule]

2017-08-25T00:52:37,274 INFO [main] io.druid.initialization.Initialization - Adding local file system extension module [io.druid.server.lookup.namespace.NamespaceExtractionModule] for class

[io.druid.initialization.DruidModule]

It seems the S3FirehoseDruidModule got registered. But still the StaticS3FirehoseFactory is not invoked. Do you know if I should change json format in the specification?

Thanks