Druid Quickstart Ingestion Failure

Hi, I am running the Druid quick start process described on (http://druid.io/docs/0.9.1.1/tutorials/quickstart.html). So far, I have it set up and am currently on the step where I attempt to batch ingest.

When I submit “curl -X ‘POST’ -H ‘Content-Type:application/json’ -d @quickstart/wikiticker-index.json localhost:8090/druid/indexer/v1/task”, I know that the task submission is successful since I receive a response of {“task”:“index_hadoop_wikiticker_2016-06-30T21:17:02.958Z”}.

However, when I proceed to the next step in the quick start tutorial, and view the task status on my overlord console, I see that the task returns with a FAILED statusCode, null as the location host and a -1 for location port. I was wondering if I could get some advice about what this problem may be? Thank you

Hey Thomas,

If you go to your overlord UI (http://localhost:8090/console.html) you should be able to fetch the logs for that failed task (click “log (all)”). Those logs should have more clues.

Hi Gian!

Thank you so much for your reply. I am new to the platform and tried looking through the logs earlier, but haven’t gained enough familiarity to usefully construe and make sense of the clues that it provided. I know that it was caused by “java.lang.reflect.InvocationTargetException” and “com.metamx.common.ISE: Job[class io.druid.indexer.IndexGeneratorJob] failed!”. Do you have advice at this stage? Thank you again for your help!

Hey Thomas,

That’s a pretty vague error, do you see anything more specific in the logs, like an exception stack trace?

Hi Gian,

Yes. An exception is raised here:

2016-06-30T21:14:12,071 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[HadoopIndexTask{id=index_hadoop_wikiticker_2016-06-30T21:14:02.954Z, type=index_hadoop, dataSource=wikiticker}]
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException

Hi Thomas, can you include the full stack trace? The actual cause is usually at the bottom.

Hi Thomas,

You may also want to look at the log entries before that InvocationTargetException, the actual exception or error that caused the task failure may be logged somewhat earlier as well.

Hi Gian, and what can i do if after clicking “log (all)” all i can see is: “No log was found for this task. The task may not exist, or it may not have begun running yet.”

The reason was that the folder was mounted to windows and the colon sign in the folder name was generating a protocol error
java.io.IOException: Could not create directories: var/druid/task/index_hadoop_wikiticker_2017-09-21T14**:26:**49.736Z/a80593c7-4174-461c-b541-757afa3a06ac

Hey, I seem to have encountered this problem too, not very sure, the following is some information on overlord, can anyone tell me how to solve it?

(I am doing ingesting data from local, curl -X ‘POST’ -H ‘Content-Type:application/json’ -d @wikiticker-index.json my_overlord_ip:8090/druid/indexer/v1/task)

payload:
{“task”:“index_wikipedia_2019-01-03T02:08:53.359Z”,“payload”:{“type”:“index”,“id”:“index_wikipedia_2019-01-03T02:08:53.359Z”,“resource”:{“availabilityGroup”:“index_wikipedia_2019-01-03T02:08:53.359Z”,“requiredCapacity”:1},“spec”:{“dataSchema”:{“dataSource”:“wikipedia”,“parser”:{“type”:“string”,“parseSpec”:{“format”:“json”,“dimensionsSpec”:{“dimensions”:[“channel”,“cityName”,“comment”,“countryIsoCode”,“countryName”,“isAnonymous”,“isMinor”,“isNew”,“isRobot”,“isUnpatrolled”,“metroCode”,“namespace”,“page”,“regionIsoCode”,“regionName”,“user”,{“name”:“added”,“type”:“long”},{“name”:“deleted”,“type”:“long”},{“name”:“delta”,“type”:“long”}]},“timestampSpec”:{“column”:“time”,“format”:“iso”}}},“metricsSpec”:,“granularitySpec”:{“type”:“uniform”,“segmentGranularity”:“DAY”,“queryGranularity”:{“type”:“none”},“rollup”:false,“intervals”:[“2015-09-12T00:00:00.000Z/2015-09-13T00:00:00.000Z”]},“transformSpec”:{“filter”:null,“transforms”:}},“ioConfig”:{“type”:“index”,“firehose”:{“type”:“local”,“baseDir”:"/home/druid/druid-0.13.0/quickstart/tutorial",“filter”:“wikiticker-2015-09-12-sampled.json.gz”,“parser”:null},“appendToExisting”:false},“tuningConfig”:{“type”:“index”,“targetPartitionSize”:5000000,“maxRowsInMemory”:25000,“maxBytesInMemory”:0,“maxTotalRows”:null,“numShards”:null,“partitionDimensions”:,“indexSpec”:{“bitmap”:{“type”:“concise”},“dimensionCompression”:“lz4”,“metricCompression”:“lz4”,“longEncoding”:“longs”},“maxPendingPersists”:0,“buildV9Directly”:true,“forceExtendableShardSpecs”:true,“forceGuaranteedRollup”:false,“reportParseExceptions”:false,“pushTimeout”:0,“segmentWriteOutMediumFactory”:null,“logParseExceptions”:false,“maxParseExceptions”:2147483647,“maxSavedParseExceptions”:0}},“context”:{},“groupId”:“index_wikipedia_2019-01-03T02:08:53.359Z”,“dataSource”:“wikipedia”}}

``

status:
{“task”:“index_wikipedia_2019-01-03T02:08:53.359Z”,“status”:{“id”:“index_wikipedia_2019-01-03T02:08:53.359Z”,“type”:“index”,“createdTime”:“2019-01-03T02:08:53.371Z”,“queueInsertionTime”:“1970-01-01T00:00:00.000Z”,“statusCode”:“FAILED”,“status”:“FAILED”,“runnerStatusCode”:“WAITING”,“duration”:4254,“location”:{“host”:null,“port”:-1,“tlsPort”:-1},“dataSource”:“wikipedia”,“errorMsg”:null}}

``

log (all):
No log was found for this task. The task may not exist, or it may not have begun running yet.

``

在 2017年9月21日星期四 UTC+8下午6:45:19,Rodrigues da Silva写道:

Hey, I seem to have encountered this problem too, not very sure, the following is some information on overlord, can anyone tell me how to solve it?
(I am doing ingesting data from local, curl -X ‘POST’ -H ‘Content-Type:application/json’ -d @wikiticker-index.json my_overlord_ip:8090/druid/indexer/v1/task)

How are you starting/configuring the services? There may be some configuration error with the cluster.

I would recommend starting fresh with the current 0.13.0-incubating quickstart (http://druid.io/docs/latest/tutorials/index.html) which has scripts for running the cluster and submitting an ingestion task, and trying the native batch ingestion tutorial (http://druid.io/docs/latest/tutorials/tutorial-batch.html).