Slow datasource availability after databse startup

Hi all,
I have a druid installation used for testing, using tutorial configuration files, with only one datasource with about 3260000 records with this segments:

13,039 segments in 735intervals

I’m using Derby as metadata store.

When a restart Druid database, only after 8-10 minutes my datasource is full available, that is it takes a lot of time before I can query datasource.

Is it normal ?

What it depends on ?

Thanks.

If you are restarting historical, then yes, it takes a bit to load your segments.

What’s your vm.max_map_count value you have set on your server? Also what does the memory config look like?

It’s a development server, and now I use druid quickstart configuration with a little bit customization. Linux parameter vm.max_map_count is 65530 and JVM config for historical is:

-server
-Xms2g
-Xmx2g
-XX:MaxDirectMemorySize=1280m
-Duser.timezone=UTC
-Dfile.encoding=UTF-8
-Djava.io.tmpdir=var/tmp
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager

and this is runtime.properties

druid.service=druid/historical
druid.plaintextPort=8083

HTTP server threads

druid.server.http.numThreads=9

Processing threads and buffers

druid.processing.buffer.sizeBytes=256000000
druid.processing.numThreads=2

Segment storage

druid.segmentCache.locations=[{“path”:“var/druid/segment-cache”,“maxSize”:300000000000}]
druid.server.maxSize=300000000000

Hi Agostino,

You might try setting maxSegmentsInNodeLoadingQueue property to speed up segment loading process. This a dynamic coordinator configuration which can be changed from the coordinator console. This doesn’t need restarting coordinators. Doc http://druid.io/docs/latest/configuration/index.html#dynamic-configuration.

Thanks,

Sashi