We’ve set up an intermediary service, that receives druids metrics and alerts configured from monitoring, and republishes metrics to stasd/graphite and alerts to sentry.
However, from what we’ve noticed, the intermediary service takes huge amount of RAM and it turns out, it receives up to 1GB of data per request! Majority of this data are alerts regarding SegmentLoadingException on historical.
While we’re aware of the problems there, is there a way to actually fine-tune what we publish in terms of alerts, or how often does druid re-publishes them on monitoring?
Our monitoring configuration for historicals: