Druid batch ingestion very small spill to disk

I have a 529mb csv file that I am batch ingesting
xms/xmx is 1/2gb
direct memory 4g
max rows in mem=1m
max bytes in mem -1
segment by day , _time has values like 2021-02-02T00:00:00:000Z
but I see lot of messages in log with spill to disk in very small numbers
here is a smaple line spill[0] to disk in [5] ms [175] rows
this is causing the ingestion to take more than 20mins
running in 1 task only
any pointer would help in speeding up the ingestion

Hey Ranjan,
what is your maxRowsPerSegment in your ingestion spec?

Hi @vsevall ,
“maxRowsPerSegment”:5000000,
the file size is 1/2 gig with 4-5 million rows
I have tried both csv and parquet formats