How to use Indexed data from s3 in a new druid cluster?

Hi all,

I have working druid server ( all services in a single box ), indexing new data daily.
It uses local storage only. and derby as metadata store.

But syncing the indexed data (var/druid/segments) to s3 daily.

Now, I want to try druid cluster.
Installed druid in 2 new servers.

In both the servers, mentioned the s3 details as deep storage.
historical, middlemanager in one server.
broker, overlord, coordinator in another server.

Started the services.

Now, If I query for any data using the co-ordinator, i get only empty results.

http://druid.io/docs/latest/design/coordinator.html

The queries here does not give any results except /status.

How to debug this?
Why the indexed data in s3 is not available to the new druid servers.

druid.s3.accessKey,
druid.s3.secretKey, ``druid.storage.bucket

The config for s3 are correct.

`Help to solve this.

Thanks.
Shrini

`

Hi Shrini,
Just copying the segments to S3 is not enough. You will also need to update the segment metadata entries in druid_segments table and point them to the s3 locations and type in loadSpec, for S3 the loadSpec type will be s3_zip.

Also, If the data is just few days, It may be easier to just reIndex the data in your new cluster which directly indexes data to s3.

Hi Nishant,

Just copying the segments to S3 is not enough. You will also need to update
the segment metadata entries in druid_segments table and point them to the
s3 locations and type in loadSpec, for S3 the loadSpec type will be s3_zip.
Also, If the data is just few days, It may be easier to just reIndex the
data in your new cluster which directly indexes data to s3.

Thanks for the details.

I reindexed all the data and built a new cluster with s3 and mysql.