Top Three Generic Performance Tips

A community member recently asked about improving performance, and received the following in reply from a founder:

  1. ensure your setup is configured well for your server. the best way to tell if this is done well is to look at your system metrics. when you’re doing a load test, the server should be running at 100% CPU (meaning it’s fully utilized). if you’re looking for subsecond queries, you want to be minimizing disk i/o, so ensure disk i/o is also low. you also want time spent doing GC to be relatively low-- a few seconds per minute at most. (you can see this by adding +XX:+PrintGC to your jvm.config or by using jvm tools.) if any of these are not looking ideal, you may need to adjust settings.
  2. ensure you don’t have too many small segments: overheads per-segment can be high in this case. you can use the web console or sys.segments table to see the average number of rows per segment. Generally we target a few million rows per segment. Lower is OK if you have a smaller amount of data. But you don’t want tons of segments that are, like, 10000 rows each
  3. you can use a flame graph (https://support.imply.io/hc/en-us/articles/360033747953-Profiling-Druid-queries-using-flame-graphs) to see what specific code is taking up your processing time. very useful tool!

You can read detailed context here, but the tips are generic and should be considered whenever you are tuning your cluster.

1 Like

Loving these performance-related shares…