I have a datasource that I have set to be a custom tier that I call ‘cold_tier’; which I have set up in the retention policy. What I am noticing is that there are segments of that datasource that are residing in the default tier historical node.
When I look at a specific segment that is stored on the _default_tier, I see that it is also replicated on the cold_tier as well… as a matter of fact, I have set this datasource to have a replication factor of 2 and is stored on 2 cold_tier historical nodes and 1 _default_tier node (that is a replication of factor 3 not 2).
Here is my questions:
Why are the segments going to _default_tier, when the retention policy states for it to goto cold_tier?
The replication factor is set to 2 for this datasource, but it seems be ignoring that because I am assuming that is due to the _default_tier?
How do I have my datasource not store data on the _default_tier?
I have attached screen shots of the retention policy along with screenshots of the services and and query I ran to verify which host the data resides on. Let me know if I am sending you the wrong things.
I think what is happening here is that the loadForever rule, which is not seen, but is still invoked is happening. I haven’t had a chance to play around with it, but I think you will need to add rules to make sure the cold data is removed from the default tier.
I see the cluster _default retention rules being reachable from every datasource retention policy, but I assume that it is a global apply i.e. there is only one _default retention, not a _default retention per datasource.
