Is there a way to add callback hooks to Druid middleManager tasks

I want to know is there a way Druid can call back an endpoint I specify after ingestion is done.

We have a spark based etl pipeline at the end of which, we upscale middleManager nodes (our whole infra is on AWS so we have a lambda to set different ASG parameters). I want to downscale after the ingestion is done.

I already have a polling based solution. I wanted to know if there is callback way to avoid polling. Please let me know if there are any ideas around it.

Hi Karthik - you can use AWS auto scaling with your cluster in Druid. I will look for some sample config and send soon.


Search for autoscaling in the below configuration reference. This can be enabled for your middle managers.


Eric Graham

Solutions Engineer -** **Imply

**cell: **303-589-4581