Druid containers crashing intermittently

I have Druid 20.2 installed with Docker containers on my local machine, which is a 2019 mac book pro with 32 GB RAM. Coordinator and Overlord are on a shared container. Historical, Broker, Middle manager each have a container of their own. Lately, the Historical and Broker containers have been restarting, seemingly spontaneously. There is no error or message I can spot in the logs. It is causing the pipeline to fail intermittently. It seems to happen around ingestion time. Any suggestions on where to look for answers or how to resolve?

Just a thought: Kubernetes will kill pods that exceed their memory limit. Check if the memory configuration each process is aligned with the memory allocation for the pod in Kubernetes.

Thank you for the reply. This is a local instance. We are not using Kubernetes.

Hello I had the same issue at first, I think its an issue due to memory allocation in Docker. I have to give at least 11 GB of ressource (change this in your Docker preferences).

Just to precise: I run Kafka and Zookeeper with all of the Druid’s components that’s why I need 11 GB :wink:

2 Likes