Tranquility - different datasources, each having different dimensions

Hi,

I am trying to leverage tranquility with kafka hose to ingest data into datasources in real time,

I have this use case where there are n datasources(can increase on a regular basis), apart from a common set of dimensions, these datasources have their own unique set of dimensions respectively. This unique set may differ completely for every data source, some new dimensions may be added to this set over time, thereby I am fine with defining ‘’ when ingesting data rather than specifying a data type.

So, the issue is,

when i ingest data with ‘’ as dimensions in dimensionSpec, I am unable to specify datatypes for the common set

when i ingest data with ‘dimensions specified’ in dimensionSpec, I am unable to specify datatypes that add over time

Can you please suggest how would I proceed to solve this?