load data from local file system

Dear All,
When loaded data from local file system using index task, I encountered the error:

java.lang.IllegalArgumentException: Parameter ‘directory’ is not a directory

I am running Druid on cluster mode, and I have one overlord node and 2 data nodes.

From the task’s log, I can see that the index task is running on node-2. While I submit the task on node-1, and expect the task to run on node-1, too. Because the data is on node-1’s local file system. Node-2 doesn’t have this data.

So, I think this can explain why I encountered the error before.

While, my question is, is there anyway I can assign the index task to run on node-1?

Because the data size is very small, so I don’t want to start hadoop job to load the data.

Thanks.

Yufeng Wang

Hi,
Ideally you should use a NFS mount to make your data available on all worker nodes.

As a workaround, yes you can specify an affinity between dataSource and worker.

Refer http://druid.io/docs/latest/configuration/indexing-service.html for more details.