Request help creating 2 specs (different granularity) for same topic

Hi,

I am using 2 tranquility nodes running below specs - where there are 2 specs in dataSoruces , everything same except granularity and dataSource.

  • spec product_view_stats(hour granularity) spawns 2 tasks (as replicas 2) but product_view_stats_total (DAY granularity) related tasks never starts why is this ?

what am I doing wrong here ?

Basically my idea is to use smaller granularity for recent 7 days and bigger granularity one for longer days (which can go for year as well).

Using above 2 specs so that I can with load and drop rules with historical node.

Is it good idea to do so ?

{

“dataSources”: [

{

“spec”: {

“dataSchema”: {

“granularitySpec”: {

“queryGranularity”: “hour”,

“type”: “uniform”,

“segmentGranularity”: “day”

},

“dataSource”: “product_view_stats_total”,

“parser”: {

“type”: “string”,

“parseSpec”: {

“format”: “json”,

“flattenSpec”: {

“useFieldDiscovery”: true,

“fields”: [

{

“type”: “path”,

“name”: “product_id”,

“expr”: “$.product.id”

},

{

“type”: “path”,

“name”: “user_id”,

“expr”: “$.user.id”

},

{

“type”: “path”,

“name”: “timestamp”,

“expr”: “$.eventTime”

}

]

},

“dimensionsSpec”: {

“dimensions”: [

“product_id”,

“user_id”

]

},

“timestampSpec”: {

“format”: “auto”,

“column”: “timestamp”

}

}

},

“metricsSpec”: [

{

“type”: “count”,

“name”: “product_view_count”

}

]

},

“tuningConfig”: {

“type”: “realtime”,

“intermediatePersistPeriod”: “PT1M”,

“windowPeriod”: “PT1M”,

“maxRowsInMemory”: 10

}

},

“properties”: {

“task.partitions”: “1”,

“task.replicants”: “2”,

“topicPattern”: “product_view_stats”,

“topicPattern.priority”: “1”

}

},

{

“spec”: {

“dataSchema”: {

“granularitySpec”: {

“queryGranularity”: “thirty_minute”,

“type”: “uniform”,

“segmentGranularity”: “hour”

},

“dataSource”: “product_view_stats”,

“parser”: {

“type”: “string”,

“parseSpec”: {

“format”: “json”,

“flattenSpec”: {

“useFieldDiscovery”: true,

“fields”: [

{

“type”: “path”,

“name”: “product_id”,

“expr”: “$.product.id”

},

{

“type”: “path”,

“name”: “user_id”,

“expr”: “$.user.id”

},

{

“type”: “path”,

“name”: “timestamp”,

“expr”: “$.eventTime”

}

]

},

“dimensionsSpec”: {

“dimensions”: [

“product_id”,

“user_id”

]

},

“timestampSpec”: {

“format”: “auto”,

“column”: “timestamp”

}

}

},

“metricsSpec”: [

{

“type”: “count”,

“name”: “product_view_count”

}

]

},

“tuningConfig”: {

“type”: “realtime”,

“intermediatePersistPeriod”: “PT1M”,

“windowPeriod”: “PT1M”,

“maxRowsInMemory”: 10

}

},

“properties”: {

“task.partitions”: “1”,

“task.replicants”: “2”,

“topicPattern”: “product_view_stats”,

“topicPattern.priority”: “1”

}

}

],

“properties”: {

“zookeeper.connect”: “10.140.0.29:2181,10.140.0.30:2181,10.140.0.31:2181”,

“zookeeper.timeout”: “PT20S”,

“druid.selectors.indexing.serviceName”: “druid/overlord”,

“druid.discovery.curator.path”: “/druid/discovery”,

“kafka.zookeeper.connect”: “10.140.0.29:2181,10.140.0.30:2181,10.140.0.31:2181”,

“kafka.group.id”: “stage5”,

“consumer.numThreads”: “2”,

“commit.periodMillis”: “15000”,

“reportDropsAsExceptions”: “false”

}

}