Druid on AWS X1 instances


how about Amazon’s X1 instances with 128 cores and almost 2 TB RAM - would they be any good for running historicals on them?

Currently we are using r3.8xlarge instances, but we almost use none of the disk space and could use more RAM. Unfortunately we also need lots of CPU cores and the CPU to RAM ratio of X1 might be a bit too low, but it would be really nice to have that much RAM and keep months of data in an X1 hot tier with 1:1 mem to disk ratio.

Has anyone given this any thought or has tried out X1 or is planning to do so?



I don’t see any reason why it wouldn’t work very well. Those machines are huge.

I know many OLAP db use instances similar to this in production (HANA comes to mind). I think it would make a great hot tier myself.

https://groups.google.com/d/msg/druid-user/l5grbWf2x9w/jD6qLln_AAAJ That is a great thread to check out.

Also be aware the X1 instances use ENA networking: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking-ena.html

By all means please report back on your experiences.