how to select all raw data from a druid datasource. I tried using select query, however it includes a threshold parameter. We have a use case where we want to pull data from druid into tableau using extracts. I could provide a huge threshold, but that api call timed out. could you please guide. I couldn’t find a solution anywhere yet.
have you tried select query with pagination ?
Check out what PlyQL (https://github.com/implydata/plyql) does when you issue a “SELECT * FROM …” query with no limit. You can see what queries it is making by running it with the --verbose flag. The results need to be paginated (which is what PlyQL does) but there is some tricks you can play with the threshold to make it much more efficient.
Thank you, Vadim.
I tried with plyql, with verbose enabled just now. I tried with different data sources, the only variation in the query is the dimensions and metrics. Threshold is set to 50 in both cases. I could get all the data where result set is around 20K. But, with a datasource of 80 mil, it times out. Is the only alternative, running query with different time intervals. Data set spans few days.
Nishant, Yes, I tried select query with pagination.