[druid-user] ISSUE Native batch performance- ingestion throttled too many times

How to reduce throtteling for Native batch ingestion task, it consumed too much amount of time and delays the processing by few hours…

2021-04-28T13:35:46,241 INFO [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Flushing in-memory data to disk because No more rows can be appended to sink,bytesCurrentlyInMemory[2650927014] is greater than maxBytesInMemory[2650920277].
2021-04-28T13:36:08,643 INFO [[partial_index_generate_RAN_eNodeB_Hourly_fjmgjmil_2021-04-28T12:53:48.307Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Flushed in-memory data for segment[RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z] spill[27] to disk in [50,832] ms (98,145 rows).
2021-04-28T13:36:11,473 INFO [[partial_index_generate_RAN_eNodeB_Hourly_fjmgjmil_2021-04-28T12:53:48.307Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Flushed in-memory data with commit metadata [null] for segments: RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z
2021-04-28T13:36:11,474 INFO [[partial_index_generate_RAN_eNodeB_Hourly_fjmgjmil_2021-04-28T12:53:48.307Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Segment[RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z] hydrant[FireHydrant{queryable=RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z, count=27}] already swapped. Ignoring request to persist.
2021-04-28T13:36:11,474 WARN [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Ingestion was throttled for [25,230] millis because persists were pending.
2021-04-28T13:36:39,077 INFO [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Flushing in-memory data to disk because No more rows can be appended to sink,bytesCurrentlyInMemory[2650940716] is greater than maxBytesInMemory[2650920277].
2021-04-28T13:37:02,537 INFO [[partial_index_generate_RAN_eNodeB_Hourly_fjmgjmil_2021-04-28T12:53:48.307Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Flushed in-memory data for segment[RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z] spill[28] to disk in [51,062] ms (98,147 rows).
2021-04-28T13:37:05,546 INFO [[partial_index_generate_RAN_eNodeB_Hourly_fjmgjmil_2021-04-28T12:53:48.307Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Flushed in-memory data with commit metadata [null] for segments: RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z
2021-04-28T13:37:05,547 INFO [[partial_index_generate_RAN_eNodeB_Hourly_fjmgjmil_2021-04-28T12:53:48.307Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Segment[RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z] hydrant[FireHydrant{queryable=RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z, count=28}] already swapped. Ignoring request to persist.
2021-04-28T13:37:05,547 WARN [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Ingestion was throttled for [26,467] millis because persists were pending.
2021-04-28T13:37:33,749 INFO [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Flushing in-memory data to disk because No more rows can be appended to sink,bytesCurrentlyInMemory[2650929440] is greater than maxBytesInMemory[2650920277].
2021-04-28T13:37:56,569 INFO [[partial_index_generate_RAN_eNodeB_Hourly_fjmgjmil_2021-04-28T12:53:48.307Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Flushed in-memory data for segment[RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z] spill[29] to disk in [51,021] ms (98,148 rows).
2021-04-28T13:37:59,434 INFO [[partial_index_generate_RAN_eNodeB_Hourly_fjmgjmil_2021-04-28T12:53:48.307Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Flushed in-memory data with commit metadata [null] for segments: RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z
2021-04-28T13:37:59,436 INFO [[partial_index_generate_RAN_eNodeB_Hourly_fjmgjmil_2021-04-28T12:53:48.307Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Segment[RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z] hydrant[FireHydrant{queryable=RAN_eNodeB_Hourly_2021-04-27T10:00:00.000Z_2021-04-27T11:00:00.000Z_2021-04-28T10:15:01.511Z, count=29}] already swapped. Ignoring request to persist.
2021-04-28T13:37:59,436 WARN [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Ingestion was throttled for [25,683] millis because persists were pending.

The druid version being used is 0.18.1.