I am using Tranquility to push events in realtime to Druid hence i wanted to measure the performance of the indexing task in terms of total number of events indexed by a task and also rate of events ingestion.
How can i get these metrics?
And also i tried setting the replication factor to 2 in the tranquility task specs but only 1 of the 2 replicas was executing and the other was always in pending state.
Do i need to setup multiple “overlord” services to use a replication factor >1?
My current setup has 2 middle managers,1 historical,1 coordinator and 1 overlord service.
My current event ingestion rate is 30,000 events per sec,all of which is time series data which implies at any point of time only 1 indexing task will get all the events.
Event size is very small and a typical 15 minute segment’s size is just 2-3 MB.
Should i index all these events with single indexing task or should i shard it to improve the throughput?