Broker cannot see datasource.

I’ve just set up a druid cluster (0.9.0) on AWS and am having trouble getting my broker to see my datasource. The coordinator console also thinks it needs to load in 100% of the data, even though I’ve had index jobs successfully complete and the data shows in deep storage (S3).

We are using separate servers for ZK, Coordinator, Overlord, MiddleManager, Broker, and a single historical node. No realtime nodes. Our deep storage is in S3 and we’re using mysql for metadata.

common.runtime.properties:

druid.extensions.loadList=[“druid-kafka-eight”, “druid-s3-extensions”, “druid-histogram”, “druid-datasketches”, “druid-namespace-lookup”, “mysql-metadata-storage”]

druid.startup.logging.logProperties=true

druid.zk.service.host=10.0.1.252

druid.zk.paths.base=/druid

druid.metadata.storage.type=mysql

druid.metadata.storage.connector.connectURI=jdbc:mysql://[route]:3306/[db_name]

druid.metadata.storage.connector.user=[user]

druid.metadata.storage.connector.password=[password]

druid.storage.type=s3

druid.storage.bucket=[bucket_name]/test

druid.storage.archiveBucket=[bucket_name]/test-archive

druid.s3.accessKey=[s3_key]

druid.s3.secretKey=[s3_secret]

druid.indexer.logs.type=s3

druid.indexer.logs.s3Bucket=[bucket_name]/index-logs

druid.indexer.logs.s3Prefix=druid/indexing-logs

druid.selectors.indexing.serviceName=druid/overlord

druid.selectors.coordinator.serviceName=druid/coordinator

druid.monitoring.monitors=[“com.metamx.metrics.JvmMonitor”]

druid.emitter=logging

druid.emitter.logging.logLevel=info

``

broker properties:

druid.service=druid/broker

druid.host=10.0.1.143

druid.port=8080

HTTP server threads

druid.broker.http.numConnections=5

druid.server.http.numThreads=25

Processing threads and buffers

druid.processing.buffer.sizeBytes=536870912

druid.processing.numThreads=1

Query cache

druid.broker.cache.useCache=true

druid.broker.cache.populateCache=true

druid.cache.type=local

druid.cache.sizeInBytes=2000000000

``

I realize this is kind of vague, but I’m not seeing any errors other than the empty set when hitting the broker_server/druid/v2/datasources endpoint.

Let me know if I can provide anything else to help diagnose the issue.

Thanks,

Spencer

Hi spencer,
Broker only sees the segments which are loaded on historical nodes.
In your case since the coordinator is showing it still has to load the data, it is most likely an issue with loading of segments on the historical nodes.
Make sure you have historical nodes running and have enough capacity to load the segments.
Also check for any exceptions related to segment loading in coordinator/historical node logs.

Just to clarify, the broker will see segments on realtimes and historicals, so I am not 100% sure what you mean when you say the broker can’t see the data. But definitely there is something wrong with handoff in your cluster.