Encountered exception while running task. java.lang.RuntimeException: Max parse exceptions exceeded, terminating task

Hi,the friends in community:

I am ingesting kafka streaming data by druid. But I found the task was failed.The log is in the attachment.

I found the error in the log:

ERROR [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Encountered exception while running task.

java.lang.RuntimeException: Max parse exceptions exceeded, terminating task…

And my kafka index spec is

{

“type”: “kafka”,

“dataSchema”: {

“dataSource”: “Parking”,

“parser”: {

“type”: “string”,

“parseSpec”: {

“format”: “json”,

“timestampSpec”: {

“column”: “endTime”,

“format”: “auto”

},

“dimensionsSpec”: {

“spatialDimensions”: [{

“dimName”: “coordinator”,

“dims”: [“lat”, “long”]

}],

“dimensions”: [

“id”,

“device_id”,

“startTime”,

{“type”: “double”,

“name”: “stayHour” },

{“type”: “double”,

“name”: “stationDistance” },

{“type”: “double”,

“name”: “logisticsParkDistance” },

“dt”,

“vin”,

“series”,

“seriesText”,

“cabCode”,

“cabTypeAdj”,

“dealerCode”,

“dealerName”,

“dealerProvince”,

“dealerCity”,

“mali”,

“vDriveFrom”,

“vDriveFormText”

]

}

}

},

“metricsSpec” : [ {

“type”: “count”,

“name”: “count”

}

],

“granularitySpec”: {

“type”: “uniform”,

“segmentGranularity”: “DAY”,

“queryGranularity”: “DAY”,

“rollup”: true

}

},

“tuningConfig”: {

“type”: “kafka”,

“reportParseExceptions”: true

},

“ioConfig”: {

“topic”: “Cross_District”,

“replicas”: 1,

“taskDuration”: “PT10M”,

“completionTimeout”: “PT20M”,

“consumerProperties”: {

“bootstrap.servers”: “10.65.254.25:9092,10.65.254.14:9092,10.65.254.58:9092”

},

“taskCount”: 1,

“useEarliestOffset”: true

}

}

The log is in the attahment.

index_kafka_Parking_73c174868a9f249_aifpnlal (118 KB)

“Max parse exceptions exceeded, terminating task” indicates that enough number of lines couldn’t be parsed and hence the task has failed ,This generally means something about your parser and/or timestampSpec doesn’t match the data.

Could you please compare the ingestion spec against some sample kafka input streaming data? to make sure your specs are valid for the data and vice versa.

Thanks ,
Vaibhav

Dear Vaibhav:
I have found I made a stupid mistake. The topic name was wrong. Thank you.

Scoffi

在 2019年10月17日星期四 UTC+8下午6:47:16,Vaibhav Vaibhav写道: