i m using curl command to do druid ingestion forcsv datasource.
the command runs successfully and a jdbc success query is shown as output in the cmd.
however the datasource is not reflecting in the druid browser local host.
I accessed druid datasources from dsql there also the datasource is not shown
CURL COMMAND USED:- curl -X ‘POST’ -H ‘Content-Type:application/json’ -d @wikipedia-index.json http://127.0.0.1:8090/druid/indexer/v1/task
someone please help if you have any idea about this issue.
What is the task status on the overlord console ?
Do you see any errors in the Overlord, Middle Manager logs ?
i am running this ingestion command on date 20/06/2019
the latest logs shown in both middlemanager.log and overload log is of date 4/06/2019.
should they have the latest runned command logs?
is there any other way to know what’s the issue ?
Also, please give a brief description about … middle manager and overload log files . these files are storing logs for what ?
It seems odd to me that the overlord would return a response to your curl request but that you wouldn’t see any recent entries in the overlord logs. Can you make sure that you’re looking at the correct log location?
Regarding why the Wikipedia data isn’t being ingested - ingestion tasks generate their own log files that you can access from one of the consoles. If you are using Druid 0.14.0+, there is a new web console available at port 8888 (by default). If you are using a prior version, you would look at the overlord console at port 8090.
If you aren’t too attached to your current deployment, I would suggest starting fresh with the latest Druid version and closely follow the quickstart here: https://druid.apache.org/docs/latest/tutorials/index.html followed by the data loading tutorial here: https://druid.apache.org/docs/latest/tutorials/tutorial-batch.html. The log files not being written make me suspect that something might be off about your deployment and that you’d have a better time starting fresh.