Why druid.server.http.numThreads should be "slightly" higher than druid.broker.http.numConnections?


The documentation for cluster tuning mentions:

druid.server.http.numThreads on the Broker should be set to a value slightly higher than druid.broker.http.numConnections on the same Broker.


Why is this so? My understanding from some responses by David is that druid.broker.http.numConnections should be such that druid.server.http.numThreads on available historicals should be higher than sum of this value across brokers which makes sense. But how does druid.server.http.numThreads affect query processing (or rather concurrent query processing if it does) on the broker node?

The out of the box configuration for the values is :

HTTP server settings


HTTP client settings



Why is the value for druid.server.http.numThreads on broker node is Way higher than druid.broker.http.numConnections?

Does number of CPUs have any connection with druid.broker.http.numConnections? How to best arrive at exact value for this depending on available compute resources, number of historicals and druid.broker.http.numConnections and druid.server.http.numThreads?


The following section in the cluster tuning guide has more details on this area: https://druid.apache.org/docs/latest/operations/basic-cluster-tuning.html#general-connection-pool-guidelines

If you’re using 0.15.0-incubating, the micro-quickstart single server example broker has 12/10 for numThreads/numConnections, the clustered broker example uses 60/50.