Re: [druid-user] Aggregating over dimensions during native batch ingestion

I think you should be able to do this. I don’t know if this is the issue, but it’s worth a try. When you reindex from druid, if you don’t keep a dimension or metric as-is, you might have to include it in the ioConfig. Otherwise, druid might not read it from the input source. (I think this must be a known bug somewhere, but I didn’t find it right away.) Eg, something like this for your ioconfig:

    "ioConfig": {
      "type": "index_parallel",
      "inputSource": {
        "type": "druid",
        "dataSource": "ingest_no_rollup",
        "interval": "2020-11-02T00:00:00.000Z/2020-11-04T00:00:00.000Z",
        "filter": null,
        "dimensions": ["ts","id","string_field","long_field"],
        "metrics": null
      },
      "inputFormat": null,
      "appendToExisting": false
    }
  },

where the dimensions are the ones you want to have it read from the input - not necessarily the same as the one you want in the new datasource. (Same can apply to metrics, but I grabbed this from something I was doing that only needed the dimensions to be “fixed”.