I know that this can be done with a postaggregation on a sum metric and a count metric, but it would be great if these was just an average metric type we could use. I haven’t seen any evidence that something like this exists, but averaging seems like a very common use case. Are there any plans to add a running average aggregator so we can easily extract it from our data?
Post Aggregator is the way to do it. One of the negative side-effects
of an "average" aggregator is that if you compute multiple of them,
you actually maintain redundant copies of the count. With a post agg,
you only have one copy.
I've always seen the post agg method and a pure "average" aggregator
as equivalent (except the corner case of running into numerical
overflow with your sums). Can you elaborate on what benefits you see
in an average aggregator a bit more?
Convenience is the only benefit I see, really. It would be nice to just have an aggregate that we could refer to as a metric in queries. It seems common enough that it shouldn’t require a post aggregator.
Talking about advantage, if we were to visualize the averages in Pivot, we need to update the configuration file which is a maintenance nightmare in general (having to maintain config file for different datasources). And for Caravel, which is sort of better than Pivot (my favorite feature being the dashboard), you have no way to defining and querying an average.
Agree, that is why we are doing away with the config files in Pivot soon (working on it now). You will get all the power of Plywood’s expressions (averages, huperUniqus, filtred aggregators, query time lookups, theta sketches) from a GUI settings page. Stay tuned
This is encouraging. Thanks for the update. I’ll keep monitoring this space, but is a there a general timeline on when we can expect it to be available ?
Hi, I am creating a proof of concept where we have, per json line, a server response time.
I want to display the average response time when I query buckets in pivot.
Would this enable me to do this? If so, when?
It is possible to set up derived measures like averages today with config files (https://github.com/implydata/pivot#create-a-config). The upcoming settings page is just about making it possible to do that from a GUI, without config files.
Just to clarify. The settings page is planned to add User Defined Measures and not fully replacing the configuration file, right?
I am asking because we’re relying heavily on the configuration file to filtering / security approaches in our company.
Do not worry. We are aware of use-cases like yours and they will be forever respected.