Is it possible to re-partition segments?

Hi,

Is it possible to reduce the number of partitions in segments using some re-partition task?

Thanks!

Michael

Hi Michael,

Yeah, you can submit a Hadoop index task to reindex those segments with your new partition setting, the new segments generated will overshadow the old ones.

If you are using Druid with a version newer than 0.8.1, Hadoop index task can take existing segments as input, check this out http://druid.io/docs/latest/ingestion/batch-ingestion.html#datasource.

Hi Bingkun,

Thanks for your answer, I was able to re-index all the segments with an updated configuration.

However, I’m seeing old segments are still present in the deep storage (s3 in our case). Are they supposed to disappear at some time?

Thanks!

Michael

Segments are forever backed up in deep storage just in case. You can use kill tasks to delete them permanently. http://druid.io/docs/latest/misc/tasks.html