Automatic segment compaction segment size not matching targetCompactionSizeBytes

Apache Druid Q&A Google Groups
#1

Hi,

I am trying to use the “Automatic segment compaction”. My auto-compaction config is as follows:

{

“dataSource”: ds,

“inputSegmentSizeBytes”:524288000,

“targetCompactionSizeBytes”: 524288000,

“skipOffsetFromLatest”: “PT3H”,

“keepSegmentGranularity”: false

}

However, after the compact task finished, I can see that the segment(shard) size is around 130+ MB. Those segments will also be involved in the next round compaction tasks and result in the same-size segments as before.

I am suspecting that tuningConfig(“maxRowsPerSegment”: 7000000) of original supervisor affected the size of the segment after compaction. Please advise on how to proceed.

Thanks.

#2

Your target compaction size is 524MB. If segment(s) that’s got a total size of less than 524MB then your segment size will be smaller and the same size.

Rommel Garcia