I am using apache druid 0.21.0 version. I am not getting Segment Granularity ALL option on the UI while ingesting data. Please refer to the below screenshot.
@Vijay_Narayanan1 @Sergio_Ferragut
Can someone help me with this? What could be the reason for not having the Segment Granularity ALL?
can you try ingesting the wikipedia dataset and see if all is available? Also please post a screenshot of the parse time tab from the above data
Please refer to the below screenshot for Wikipedia parse time.
Please refer to the below screenshot for Wikipedia segment granularity options.
I am not getting Segment Granularity ALL for the Wikipedia dataset as well.
ok. Based on what I see it seems that there is no problem. If you see your earlier screenshot all the segments are for the same time period. Hence the all granularity is being respected. Having said that, why is segment granularity ‘all’ so critical?. Segment granularity has an impact only on query performance not on results. Using ‘all’ discards the performance advantage you gain by partitioning by time
Segment Granularity ALL is required to solve the distinct count issue mentioned below
We need to show a unique exact count in our application and for now, I cannot move on to SQL query because that is required a lot of effort on the application side and our entire application is running on native queries. So for now easiest solution is to have a single segment and in that case, the distinctCount aggregator will work fine. I have already verified the performance by having 15M records in a single segment and in my case, we will not have that many records for now.
One more thing, when I am uploading the same dataset on Druid version 0.22.1 then I am getting Segment Granularity ALL option. So is it related to the version?
