Druid suddenly just close the connection to RabbitMQ

For some reasons, the realtime cluster just suddenly close the connection to RabbitMQ and not ingest the data. I got this exception

2016-02-18T18:20:27,069 ERROR [chief-sparrow-firehose-web[0]] io.druid.segment.realtime.RealtimeManager - RuntimeException aborted realtime processing[firehose-web]: {class=io.druid.segment.realtime.RealtimeManager, exceptionType=class com.metamx.common.parsers.ParseException, exceptionMessage=Unknown type[class java.util.ArrayList]}
com.metamx.common.parsers.ParseException: Unknown type[class java.util.ArrayList]
at io.druid.data.input.MapBasedRow.getLongMetric(MapBasedRow.java:157) ~[druid-api-0.3.13.jar:0.3.13]
at io.druid.segment.incremental.SpatialDimensionRowTransformer$3.getLongMetric(SpatialDimensionRowTransformer.java:135) ~[druid-processing-0.8.3.jar:0.8.3]
at io.druid.segment.incremental.IncrementalIndex$1$2.get(IncrementalIndex.java:101) ~[druid-processing-0.8.3.jar:0.8.3]
at io.druid.query.aggregation.LongSumAggregator.aggregate(LongSumAggregator.java:58) ~[druid-processing-0.8.3.jar:0.8.3]
at io.druid.segment.incremental.OnheapIncrementalIndex.addToFacts(OnheapIncrementalIndex.java:172) ~[druid-processing-0.8.3.jar:0.8.3]
at io.druid.segment.incremental.IncrementalIndex.add(IncrementalIndex.java:462) ~[druid-processing-0.8.3.jar:0.8.3]
at io.druid.segment.realtime.plumber.Sink.add(Sink.java:127) ~[druid-server-0.8.3.jar:0.8.3]
at io.druid.segment.realtime.plumber.RealtimePlumber.add(RealtimePlumber.java:224) ~[druid-server-0.8.3.jar:0.8.3]
at io.druid.segment.realtime.RealtimeManager$FireChief.runFirehose(RealtimeManager.java:362) ~[druid-server-0.8.3.jar:0.8.3]
at io.druid.segment.realtime.RealtimeManager$FireChief.run(RealtimeManager.java:264) [druid-server-0.8.3.jar:0.8.3]

``



This is the log before that

2016-02-18T18:20:21,994 INFO [firehose-web-incremental-persist] io.druid.firehose.rabbitmq.RabbitMQFirehoseFactory - Acknowledging delivery of messages up to tag: 18895100
2016-02-18T18:20:21,994 INFO [firehose-web-incremental-persist] io.druid.segment.ReferenceCountingSegment - Closing firehose-web_2016-02-18T18:00:00.000Z_2016-02-18T19:00:00.000Z_2016-02-18T18:00:00.000Z, numReferences: 0

2016-02-18T18:20:21,994 INFO [firehose-web-incremental-persist] io.druid.segment.ReferenceCountingSegment - Closing firehose-web_2016-02-18T18:00:00.000Z_2016-02-18T19:00:00.000Z_2016-02-18T18:00:00.000Z
2016-02-18T18:20:21,992 INFO [firehose-web-incremental-persist] io.druid.segment.IndexIO$DefaultIndexIOHandler - Skipped files[[index.drd, inverted.drd, spatial.drd]]
Not sure if the log helps much but it didn't show anything else apart from normal ingesting log.

Hey you have the following exception which is the cause of the runtime exception thought

Noppanit,

It looks like you have a spatial dimension referencing a field that isn’t in the expected format. Could you double-check your spatial dimension config and also your message format?

Btw, the fact that this halted ingestion rather than just incrementing the unparseable count is a bug with things that need additional parsing during indexing (like spatial dimensions). It should be fixed by https://github.com/druid-io/druid/pull/2505

Oops, nevermind, this isn’t spatial dimensions. I got tricked by the stack trace!

It’s a long aggregator pointed at a field that isn’t actually a long.

At any rate, it should still be fixed by that PR.