Approximate Histogram aggregator response doc

Hello,

I know this is an experimental feature, but is there any doc/explanation on the histogram post-aggregator responses?

The query seems to work well but we are not sure about the values and their meaning, like min and max does not seem to match our data

“aggregations”:[

{“type”:“approxHistogramFold”,“name”:“duration”,“fieldName”:“sessionDurationHistogram”}

],

“postAggregations”:[

{ “type” : “quantiles”, “name” : “durationQuantile”, “fieldName” : “duration”, “probabilities” : [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]}

],

GIVES

[

{

“timestamp”: “2016-09-14T08:54:00.000Z”,

“result”: {

“duration”: {

“breaks”: [

-241434.5,

92,

241618.5,

483145,

724671.5,

966198,

1207724.5,

1449251

],

“counts”: [

1,

29,

1,

1,

1,

0,

1

]

},

“durationQuantile”: {

“probabilities”: [

0.1,

0.2,

0.3,

0.4,

0.5,

0.6,

0.7,

0.8,

0.9

],

“quantiles”: [

5456.4,

12624.8,

13304.4,

18458.203,

27078,

48711.016,

69688.59,

123343.03,

288955

],

“min”: 92,

“max”: 1449251

}

}

}

]

max here is 1449251 (ms) but it should be much higher, like above 133200000 ms

Also why is the first break value negative? is that extraneous? since the metric is duration here it should never be negative.

Any thoughts or clarification?

Hello again,

I sorted out the max value, it is actually correct!

the only mystery now it the first breaks that is negative

Hi Mickael
Were you able to get resolution on this one, I am trying out this feature and do see negative breaks when infact my aggregation is count column and shouldn’t have negative values, can you update.

Thanks

Meera

Hi,

after a while, or with more data, values get more accurate and negative figures disappear… so we are fine with it so far

Thanks Mickael for your response.
I am not sure I understand the response that I get for my query - how can quantiles be smaller than the breaks in the histograms.

max seems to be approximately correct, rest I am not sure. what is the version of druid where you have approx histogram enabled, We are on 0.9.0

Any help will be appreciated.

thanks

meera

“granularity”: {

“timeZone”: “MST”,

“period”: “PT12H”,

“type”: “period”

},

“aggregations”: [

{

“name”: “histograms”,

“fieldName”: “count”,

“type”: “approxHistogram”

}

],

“postAggregations”: [

{

“name”: “percentiles”,

“fieldName”: “histograms”,

“type”: “quantiles”,

“probabilities”:[0.05, 0.1,0.5,0.9,0.95],

“numBuckets”:10,

“lowerLimit” : 0

}

],

as

[

{

“version”: “v1”,

“timestamp”: “2017-03-21T12:00:00.000-07:00”,

“event”: {

“pool”: “r1cart”,

“percentiles”: {

“probabilities”: [

0.05,

0.1,

0.5,

0.9,

0.95

],

“quantiles”: [

1,

1,

1,

1,

1

],

“min”: 1,

“max”: 4422

},

“histograms”: {

“breaks”: [

-735.8333129882812,

1,

737.8333129882812,

1474.6666259765625,

2211.5,

2948.333251953125,

3685.16650390625,

4422

],

“counts”: [

0,

258432,

3,

3,

6,

4,

1

]

},

“type”: “URL”

}

},

{

“version”: “v1”,

“timestamp”: “2017-03-22T00:00:00.000-07:00”,

“event”: {

“pool”: “r1cart”,

“percentiles”: {

“probabilities”: [

0.05,

0.1,

0.5,

0.9,

0.95

],

“quantiles”: [

1,

1,

14.335338,

39.438404,

48.881668

],

“min”: 1,

“max”: 162

},

“histograms”: {

“breaks”: [

-25.83333396911621,

1,

27.83333396911621,

54.66666793823242,

81.5,

108.33333587646484,

135.1666717529297,

162

],

“counts”: [

0,

200937.125,

74449.4921875,

10354.9150390625,

185.4681396484375,

4,

3

]

},

“type”: “URL”

}

}

]