How does coordinator decide which segment to assigned for a historical node to load

Hi All,

Can someone here help me with following queried, thanks for help .

When broker receives a query it finds segment information from zooserver, which segments are stored in historical and which ones are in realtime.

Does it mean that historical and realtime combined will have all the segment copy available in deep storage(except relatime)? as broker also get segments from deep storage if not present in historical.

coordinator assign segments to all historical node, does it mean any new data comes to deep storage that segment also should be present in historical.

Thanks,

Rahul

Queries only run against data actually on historicals or realtime nodes. Coordinators tell historicals tod download a segment, after which the historical will publish to zk.

Brokers never interact with deep storage.

–Eric

Thanks Eric. Does it mean all the data that is there in deep storage will also be in historical node for querying.

Hi Rahul,

Query can run only on data present in historical node and data present in historical node depends on the rule defined on specific datasources (There could be default rules as well which will be applicable to all the datasources which does not have their specific rules).

You can define you rules directly on coordinator console or you can submit it through task.

You may want to go through below thread once in detail for your understanding (attached sample screen shot of how rules page look like).

https://groups.google.com/forum/#!topic/druid-user/25MQIyqZqUI

Regards,

Arpan Khagram

+91 8308993200