I was trying to automate the deployment of druid ingestion specs for Druid.
I’m able to check if datasource in ingestion spec is already available in Druid.
I was curious if there is any way by which I can tag the ingestion spec (maybe with setting the version) in druid for comparison purpose to update the existing datasource or to skip the updation as same version of ingestion spec is already available in druid as well.
Hm I’m not aware of any metadata inside ingestion specs… it sounds like a really good idea though…!!
Oddball option: could you add a dimension that is always, say, the version of the ingestion spec? I expect it will be small if stored as a string on indexing because it will be stored just as one value for the entire column in the segment … but then you may be able to query the datasource itself to see what version of spec actually was used to ingest a row… I told you it was an oddball idea…!!
There is a
context field in ingestion spec. I am wondering whether you can use this to achieve what you want. Not sure whether this is officially recommended.
Thanks for it Joy
Will definitely look over it