Dropping columns while performing compaction

Hi All,

Would appreciate any help regarding the following :

  1. Is there a way to drop columns (dimensions or metrics) while running a compaction on segments ?

  2. Is there a way to clean up older versions of segments which have been compacted or does compaction itself takes care of cleaning it up from the deep storage and metastore.



Hey Abhinav,

For 1) not with a compaction task, but if you use a full fledged indexing task (either native indexing with ingestSegment firehose, or hadoop with dataSource input type) then you can do any adjustments you want, including dropping columns.

For 2) you can do this by running a kill task, or enabling auto-kill.