"multi" inputSpec for merging multiple datasources into one datasource

Hi all,

Tried the “multi” inputSpec for merging N datasources into one. The ingestion worked without any errors but the result is not what I’ve expected.

I have at input 2 datasources:

  • ds1 with metric clicks - value 100 after rollup
  • ds2 with metric click - value 100 after rollup

After running the “multi” ingestion specifying the 2 datasources above at input and ds3 datasource at output, I have data only from the first datasource: click metric - 100.

I expected the ingestion to aggregate the data from both input datasources and have click - 200.

Did I misunderstand the usage/reason of “multi” inputSpec feature or I did something wrong?

P.S.

That worked with the CombiningFirehose with 2 ingestSegment delegates.

The solution was to use the “rollup” : true flag at the granularitySpec level. That worked for this case.

Thanks.