Implementing custom load/drop rules to control the segments.

Hi all,

I am interested on developing a specific drop rule to control the segments. For example:

I have 2 datasources on 1 historical node. If I enable my custom drop rule, I want something similar this:

  • If I have 2 datasource, each datasource has 50% of the historical size.

  • If one of them has 20% and another has 50%, I want the second datasource to start to remove old segments and accept new segments.

Other example, the possibility to set a XXX Mbytes to a datasource and it can load only this Mbytes, when the space is full the segments rotates.

In conclusion, I want to develop a rule based on the size (% or bytes) of the historical instead of the time or intervals.

Is this possible?



Hi Andres, you can start by taking a look at the Rule interface. Rules for loading data extend I think for this use case, you may need to rework the Rule interface and specifically the appliesTo method, which is a larger proposal. I think we should continue this conversation in the druid development forum.

Hi Fangjin,

I have continued the new thread on druid development:!topic/druid-development/A8iMYOIG89U