Our brokers did not handle queries. And we changed some segments’ “used” to “0” in MySQL, as the segments decrease the brokers come back to work.
We use Kafka Indexing Service to ingest realtime data, and the segmentGranularity is 1H. Thus our segments are very small, 300KB ~ 50MB. I wander the number of segments affect the concurrency of the Broker.
So is there any recommended configuration of Broker numConnections and numThreads?
We have 2 overlords, 2 coordinators, 19 middleManagers, 8 historicals, 4 brokers(4 brokers in each physical machines, thus 16 brokers together).
Our config of Broker is as follows:
# HTTP server threads
# Processing threads and buffers
# Query cache