I have a scenario where new metrics can be added dynamically. This won’t happen very often but new metrics are “discovered” and we would like to start creating aggregates on them automatically. We are using Tranquility from Spark via the Tranquilizer class to upload data to the indexer service. When we discover new metrics we can create a new Beams, but the new metrics don’t show up in the segments. My guess is that this is because the old streaming task continues to run on the indexer node and it treats the new metrics like schema-less dimensions. How should we be handling our scenario?
Tranquility creates a task for an interval with a defined schema. As new metrics show up, they won’t reflect in the data until the next time Tranquility runs a task.