The docs seems to indicate that I can run a single overlord process for small data loads, for example on the batch ingestion doc page it only shows an overlord being started. Is this correct?
Also, let’s say I have my data stored by hour in HDFS, something like this:
The docs seem to indicate that I can use inputSpec of type “granularity” to point to a custom path on HDFS. Something like this?
I’m not really sure what to set dataGranularity to above, because the docs indicate it’s an object, but there is no documentation on what that object format is. The only hint I have is that "hour means to expect directories
Am I on the right track, and if so, what to put for dataGranularity?