Docker volumes

Hi

I have created my own druid docker image and it works fine. In the docker file I create two directories, one for binaries and configuration files (/usr/local/druid) and another for data (/var/druid)

RUN mkdir -p /usr/local/druid/lib && \

mkdir -p /usr/local/druid/extensions && \

mkdir -p /usr/local/druid/hadoop-dependencies && \

mkdir -p /usr/local/druid/conf && \

mkdir -p /var/druid/segments && \

mkdir -p /var/druid/indexing-logs && \

mkdir -p /var/druid/task && \

mkdir -p /var/druid/segment-cache && \

mkdir -p /tmp/druid

``

As druid is part of a microservices application I use dockr-compose to start the druid container

app-sb-ax-druid:

image: app-sb-ax-druid

ports:

  • “18080:8080”

  • “18081:8081”

  • “18082:8082”

  • “18083:8083”

  • “18090:8090”

volumes:

  • app-sb-ax-druid__tmp:/tmp

  • app-sb-ax-druid__var_druid:/var/druid/

container_name: app-sb-ax-druid

``

As you can see I create the volume app-sb-ax-druid__var_druid:/var/druid/ The question: after indexing some historical data sources and starting new containers with the same volume instruction the new containers “do not see” the indexed data. I thought that all the information relating to indexed data sources woud reside in the segments and segment-cache directories, but obviously I am missing something.

What should I do (in terms of docker volumes) in order to persist indexed data among containers?

Thanks for any help

Where does your metadata storage reside? Maybe the metadata database is getting discarded and recreated, it would need to persist across containers.