Druid broker returning empty response irrespective of the type of queries

Arun_B · June 20, 2018, 2:10pm

Druid coordinator - config:

Druid coordinator, overlord and zookeeper run on the same host:

{“version”:“0.12.0”,“modules”:[{“name”:“io.druid.storage.s3.S3StorageDruidModule”,“artifact”:“druid-s3-extensions”,“version”:“0.12.0”},{“name”:“io.druid.firehose.s3.S3FirehoseDruidModule”,“artifact”:“druid-s3-extensions”,“version”:“0.12.0”},{“name”:“io.druid.metadata.storage.mysql.MySQLMetadataStorageModule”,“artifact”:“mysql-metadata-storage”,“version”:“0.12.0”}],“memory”:{“maxMemory”:3087007744,“totalMemory”:3087007744,“freeMemory”:2734403792,“usedMemory”:352603952}}

Druid coordinator- log

2018-06-20T13:31:18,330 INFO [DatabaseRuleManager-Exec–0] io.druid.metadata.SQLMetadataRuleManager - Polled and found rules for 1 datasource(s)
2018-06-20T13:31:32,926 INFO [DatabaseSegmentManager-Exec–0] io.druid.metadata.SQLMetadataSegmentManager - Polled and found 25 segments in the database
2018-06-20T13:31:42,470 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorSegmentInfoLoader - Starting coordination. Getting available segments.
2018-06-20T13:31:42,471 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorSegmentInfoLoader - Found [25] available segments.
2018-06-20T13:31:42,471 INFO [Coordinator-Exec–0] io.druid.server.coordinator.ReplicationThrottler - [_default_tier]: Replicant create queue is empty.
2018-06-20T13:31:42,471 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorBalancer - [_default_tier]: One or fewer servers found. Cannot balance.
2018-06-20T13:31:42,471 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorLogger - [_default_tier] : Assigned 0 segments among 1 servers
2018-06-20T13:31:42,471 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorLogger - [_default_tier] : Dropped 0 segments among 1 servers
2018-06-20T13:31:42,471 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorLogger - Load Queues:
2018-06-20T13:31:42,471 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorLogger - Server[<historical_Node_IP>:8083, historical, _default_tier] has 0 left to load, 0 left to drop, 0 bytes queued, 1,421,809,653 bytes served.
2018-06-20T13:32:12,471 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorSegmentInfoLoader - Starting coordination. Getting available segments.
2018-06-20T13:32:12,471 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorSegmentInfoLoader - Found [25] available segments.
2018-06-20T13:32:12,471 INFO [Coordinator-Exec–0] io.druid.server.coordinator.ReplicationThrottler - [_default_tier]: Replicant create queue is empty.
2018-06-20T13:32:12,472 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorBalancer - [_default_tier]: One or fewer servers found. Cannot balance.
2018-06-20T13:32:12,472 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorLogger - [_default_tier] : Assigned 0 segments among 1 servers
2018-06-20T13:32:12,472 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorLogger - [_default_tier] : Dropped 0 segments among 1 servers
2018-06-20T13:32:12,472 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorLogger - Load Queues:
2018-06-20T13:32:12,472 INFO [Coordinator-Exec–0] io.druid.server.coordinator.helper.DruidCoordinatorLogger - Server[<historical_Node_IP>:8083, historical, _default_tier] has 0 left to load, 0 left to drop, 0 bytes queued, 1,421,809,653 bytes served.
Druid - historical - config

curl http://localhost:8083/status

{“version”:“0.12.0”,“modules”:[{“name”:“io.druid.storage.s3.S3StorageDruidModule”,“artifact”:“druid-s3-extensions”,“version”:“0.12.0”},{“name”:“io.druid.firehose.s3.S3FirehoseDruidModule”,“artifact”:“druid-s3-extensions”,“version”:“0.12.0”},{“name”:“io.druid.metadata.storage.mysql.MySQLMetadataStorageModule”,“artifact”:“mysql-metadata-storage”,“version”:“0.12.0”}],“memory”:{“maxMemory”:465765400576,“totalMemory”:36986945536,“freeMemory”:33716764816,“usedMemory”:3270180720}}

Druid - Historical - log

2018-06-19T14:20:09,020 INFO [main-EventThread] org.apache.curator.framework.state.ConnectionStateManager - State change: RECONNECTED
2018-06-19T14:20:09,023 INFO [NodeTypeWatcher[coordinator]] io.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Ignored event type [CONNECTION_RECONNECTED] for nodeType [coordinator] watcher.
2018-06-19T14:20:09,023 INFO [Announcer-0] io.druid.curator.announcement.Announcer - Reinstating [/druid/announcements/8083]
2018-06-19T14:20:09,023 INFO [ZkCoordinator] io.druid.server.coordination.ZkCoordinator - Ignoring event[PathChildrenCacheEvent{type=CONNECTION_RECONNECTED, data=null}]
2018-06-19T14:20:09,023 INFO [Announcer-0] io.druid.curator.announcement.Announcer - Reinstating [/druid/segments//.ec2.internal:8083_historical__default_tier_2018-06-19T14:09:45.802Z_2b27ae1271474d128031b8507c09ebd30]
2018-06-19T14:20:09,024 INFO [Announcer-0] io.druid.curator.announcement.Announcer - Reinstating [/druid/internal-discovery/historical/.ec2.internal:8083]
2018-06-19T14:20:09,024 INFO [Announcer-0] io.druid.curator.announcement.Announcer - Reinstating [/druid/listeners/lookups/__default/http:i.ec2.internal:8083]
2018-06-19T14:20:39,374 INFO [Announcer-0] io.druid.curator.announcement.Announcer - Node[/druid/internal-discovery/historical/.ec2.internal:8083] dropped, reinstating.
2018-06-19T14:20:39,374 INFO [Announcer-0] io.druid.curator.announcement.Announcer - Node[/druid/segments/.ec2.internal:8083/.ec2.internal:8083_historical__default_tier_2018-06-19T14:09:45.802Z_2b27ae1271474d128031b8507c09ebd30] dropped, reinstating.
2018-06-19T14:20:39,375 INFO [Announcer-0] io.druid.curator.announcement.Announcer - Node[/druid/announcements/.ec2.internal:8083] dropped, reinstating.
2018-06-19T14:20:39,375 INFO [Announcer-0] io.druid.curator.announcement.Announcer - Node[/druid/listeners/lookups/__default/http:.ec2.internal:8083] dropped, reinstating.
2018-06-19T14:20:39,375 INFO [NodeTypeWatcher[coordinator]] io.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Node[coordinator:DiscoveryDruidNode{druidNode=DruidNode{serviceName=‘druid/coordinator’, host=’.ec2.internal’, port=-1, plaintextPort=8081, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}, nodeType=‘coordinator’, services={}}] disappeared.
2018-06-19T14:20:39,389 INFO [NodeTypeWatcher[coordinator]] io.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Received event [CHILD_ADDED] for Node[coordinator:DiscoveryDruidNode{druidNode=DruidNode{serviceName=‘druid/coordinator’, host=’.ec2.internal’, port=-1, plaintextPort=8081, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}, nodeType=‘coordinator’, services={}}].

I am able to see the list of dimensions and metrics when i do a

curl http://localhost:8082/druid/v2/datasources/<datasource_name>?interval=2018-06-05T00:00:00.000Z/2018-06-06T00:00:00.000Z

but the same is returning empty when i do a

curl http://localhost:8082/druid/v2/datasources/<datasource_name>

and this is the basic query that I used to query broker

{

“queryType”: “groupBy”,

“dataSource”: “<datasource_name>”,

“granularity”: “all”,

“dimensions”: ,

“aggregations”: [

{“type”: “count”, “name”: “rows”}

],

“intervals”: [“2018-06-05T00:00:00.000Z/2018-06-06T00:00:00.000Z”]

}

``

and this query is returning empty. I have tried different types of queries but they are all returning empty responses…

GunWoo_Kim · June 30, 2018, 6:16pm

you groupBy query has empty list for “dimensions” field.

is is correct?

Arun_B · July 2, 2018, 11:09am

Hi GunWoo,

Thanks for the response. The issue was that all partitioned segments for a time interval was not loaded in Druid and hence that time interval was not queryable.