Graceful shutdown of Historical node

Good morning!

If I need to restart a Historical node for some reason, is there a way to let it serve all currently running queries and not to interrupt them? Currently our queries sometimes fail with the following error in broker logs:

exceptionType=class com.metamx.common.RE, exceptionMessage=Failure getting results from[http://XXX:8100/druid/v2/] because of [ Channel disconnected]


Nobody knows the answer?

For now we don’t have a way of doing this Sorry !

But since the data is replicated over other historical issuing the query again will be routed to the set of available historical.

Does broker repeat the query automatically? Seems not. So it is up to the client to repeat the query. And that is the main pain point here :frowning:

Right now, broker only retries a query for a segment in case the segment is moved across to another historical node before the query is served.
I think it makes sense to extend current broker retry logic to also serve query from a replica in case one historical node goes down.

could you submit a github issue or PR for this ?