We are lining up our batch segments against our real-time segments (which are produced via Tranquility). When looking at multi-valued fields (in pivot), we noticed that the segments produced by real-time contain “null” values, whereas the segments produced by our batch processes do not.
If possible, I’d like to eliminate that “null” value from the real-time segments to exactly match what is produced by batch.
Via a Tranquilizer, we are sending a Map<String,Object> as our event object. Each entry in the map is a dimension. For the multi-valued dimensions, we supply a Set as our key. In the case, where we don’t have a value, I am presently supplying an empty set as the value. (which results in null values in Pivot)
I tried omitting the key if the event had no values for that dimension.
I also tried supplying null as the key value.
But alas, I still see null values in Pivot.
I’m trying to track this down in the code, and came to the sendBatch method on the DruidBeam class in Tranquility, but then it occurred to me that it might be the server side indexing task, and its interpretation of the values sent from Tranquility. (at which time I thought I should just ask =)
any help is appreciated,