[druid-user] Re: I am seeing a case where Druid ingestion finished in 9 seconds successfully without errors. Something is up

Hey Didip,

9 seconds is very fast considering it takes 5 seconds just for the task to start up. Was the interval correct and were there any segments in that interval?

Hmm… very interesting. Is there anything interesting in the task logs? Were there any open locks on the interval at that time?

Would it be possible to attach the ingestion spec and the main task log? It might help clarify the details.

Thanks! Btw, you don’t want to set dropExisting to true. That would mean that the segments you specified to ingest to would get dropped even if no new data were loaded into them.
Is this the same spec that was used initially to load, and it worked the first time?

“s3://mybucket/warehouse/source_schema/source_table/dt=2021-10-15” still exists and has the right data in it?

Just reading it, I’m not sure why there’s a final “dataSource”: “mytable-prod” at the end, at the top (JSON) level, or if that has any effect. Do you see that in working specs?

I had a couple of things I might try - feel free to try or not.

  1. If you ingest it into a new datasource, does it load anything?
  2. If you remove the dimensionExclusions section, does it then load anything? (That should be ignored, since you specify dimensions, but having __time in there makes me wonder.)
    Otherwise, since there are no errors, it seems like it’s reading the file but concluding nothing needs to be loaded, as you said, so all I can think of is that the date field is off.

Oh, this worked in 0.21.1, but not in 0.22.0 (or which version are you using)?

I’m glad to hear it! Excellent, thanks for following up and letting us know!