Tranquility goes out of memory

Tranquility consuming from kafka goes out of memory for high volume topic. How do I increase the memory for tranquility?

2016-04-12 14:17:56,311 [ConsumerFetcherThread-tranquility-consumer_c14-og2a-cm-druid-overlord-1.srv.media.net-1460468796515-ec381872-0-2] ERROR k.network.BoundedByteBufferReceive - OOME with size 25166359

java.lang.OutOfMemoryError: Java heap space

at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57) ~[na:1.8.0_77]

at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) ~[na:1.8.0_77]

at kafka.network.BoundedByteBufferReceive.byteBufferAllocate(BoundedByteBufferReceive.scala:80) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:63) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.network.Receive$class.readCompletely(Transmission.scala:56) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.network.BlockingChannel.receive(BlockingChannel.scala:111) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:71) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:68) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:112) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:112) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:112) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:111) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:111) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:111) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:110) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:94) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) [org.apache.kafka.kafka_2.11-0.8.2.2.jar:na]

2016-04-12 14:17:56,311 [ConsumerFetcherThread-tranquility-consumer_c14-og2a-cm-druid-overlord-1.srv.media.net-1460468796515-ec381872-0-2] INFO kafka.consumer.SimpleConsumer - Reconnect due to socket error: jav

a.lang.OutOfMemoryError: Java heap space

Hey Rishi,

You should be able to specify a heap size on the command line like this,

bin/tranquility -J-Xmx1g kafka

(or any other Xmx you like)

Thanks Gian

I’m using the Imply suite. Tranquility is started up by the supervise command.
The command line option for the JVM that I put in the conf/supervise/data.conf seems to be ignored.
The bin/tranquility loads strings from dist/tranquility/conf/application.ini

Any ideas?

thanks,

Kenji