Ingestion Metrics docs talk about realtime node config


I’m looking to have the Kafka consumer lag emitted in the logs. I’ve seen that the lag is emitted to the metric ingest/kafka/lag ( but when I check that metric in the metrics documentation it mentions that ingestion metrics ‘…are only available if the RealtimeMetricsMonitor is included in the monitors list for the Realtime node. These metrics are deltas for each emission period.’

IIRC realtime nodes are a deprecated system and we’re using the Kafka Indexing Service. So then, what node config needs updating to output the kafka topic consumer lag messages?

(I’m aware of the supervisor status, but that doesn’t fit well into our metrics systems)



be good if I actually linked the latest code correctly rather than on old fork :frowning:

Thanks for the question, our docs there will need to be updated.

For ingest/kafka/lag, the emission is not tied to a Monitor, so it should always be emitted by the supervisor.

For the other ingestion metrics like event counts, the Monitor object that handles those metrics will always be installed for Kafka Indexing Service tasks and no additional config is needed:


Reviving this thread since I have a related question: is it possible to monitor ingest/** metrics in batch indexing as well? I wasn’t able to see them when using the logging emitter and I’m not sure what extra config is required.




I hope Metrics is enable and being emitted. Are you not seeing the metrics listed under “Indexing Service” in the following?

Thanks & Regards


I can see metrics emitted. Haven’t checked if the task metrics are emitted.

I’m trying to measure the ingestion throughput, so I’m interested in the ingestion metrics listed right above “Indexing Service” in the metrics documentation. Are those available at all, or is there another method to measure ingestion throughput/latency in batch and stream indexing?