Re: [druid-user] Is it possible to use S3 as deep storage for a Druid cluster running on a non-Amazon data center?

Hi Xuan,

Yes you can use druid with s3 deep storage. I would suggest tuning your memories to keep maximum data in memory. Druid doesn’t use deep storage in real time queries . So it won’t make much difference but it will provide better scalability as you don’t have to keep taking back up of your disk when your segments will be safe in s3.

https://druid.apache.org/docs/latest/development/extensions-core/s3.html

Druid uses deep storage only as a backup of your data and as a way to transfer data in the background between Druid processes. To respond to queries, Historical processes do not read from deep storage, but instead read prefetched segments from their local disks before any queries are served. This means that Druid never needs to access deep storage during a query, helping it offer the best query latencies possible. It also means that you must have enough disk space both in deep storage and across your Historical processes for the data you plan to load.

https://druid.apache.org/docs/latest/design/architecture.html#deep-storage

Thanks & Regards,
Vikram