Re: [druid-user] Derived datasource with derived metrics in Druid

Hi Manisha,

Can you try the following ingestion spec? During query you can use “sum_fail_count/ sum_volume” to compute failure rate. Hope this meets your requirement.

{

“type”: “index_parallel”,

“spec”: {
“ioConfig”: {
“type”: “index_parallel”,
“inputSource”: {
“type”: “inline”,
“data”: “{“event_name”: “TEST_EVENT”,“properties”: { “id”: “abc123”, “type” : “TEST”, “status”: “FAILED”, “error_type” : “INTERNAL” },“event_timestamp”: 1607951231 }\n{“event_name”: “TEST_EVENT”,“properties”: { “id”: “abc123”, “type” : “TEST”, “status”: “STARTED”, “error_type” : “INTERNAL” },“event_timestamp”: 1607951241 }”
},
“inputFormat”: {
“type”: “json”
}
},
“tuningConfig”: {
“type”: “index_parallel”,
“partitionsSpec”: {
“type”: “dynamic”
}
},
“dataSchema”: {
“dataSource”: “inline_data”,
“granularitySpec”: {
“type”: “uniform”,
“queryGranularity”: “HOUR”,
“rollup”: true,
“segmentGranularity”: “HOUR”
},
“timestampSpec”: {
“column”: “event_timestamp”,
“format”: “posix”
},
“transformSpec”: {
“transforms”: [
{
“type”: “expression”,
“name”: “fail_count”,
“expression”: “if(status == ‘FAILED’,1,0)”
},
{
“type”: “expression”,
“name”: “volume”,
“expression”: “if(status == ‘STARTED’,1,0)”
}
]
},
“dimensionsSpec”: {
“dimensions”: [
“event_name”
]
},
“metricsSpec”: [
{
“name”: “count”,
“type”: “count”
},
{
“name”: “sum_fail_count”,
“type”: “longSum”,
“fieldName”: “fail_count”
},
{
“name”: “sum_volume”,
“type”: “longSum”,
“fieldName”: “volume”
}
]
}
}
}

Regards, Chari

Manisha,

To my knowledge, its not possible. You have to reindex and build new datasource to get what you are looking for.

One more thing, if you pre-compute, you wont be able to use that metric at higher levels. ex: if granularity in the segment is hour, if reporting requirement changes to day - then this metric cant just be summed. you would need to recompute it at different grain.

Lets see if anyone from the community has a solution.

Regards, Chari.

Hey Manisha!

Have you considered using an expression as a roll-up metric? Roll-up outputs a metric calculation periodically by collapsing at a given time_trunc.

See https://youtu.be/VGpPaGhuz1g?t=407