Tranquility Storm Bolt - Async loop died!

Hi,

When I run the storm topology I get the following error…

[ERROR] Async loop died!

java.lang.NoSuchMethodError: scala.collection.JavaConverters$.asScalaBufferConverter(Ljava/util/List;)Lscala/collection/convert/Decorators$AsScala;

at com.metamx.tranquility.druid.DruidRollup$.create(DruidRollup.scala:168) ~[stormjar.jar:na]

at com.metamx.tranquility.druid.DruidRollup.create(DruidRollup.scala) ~[stormjar.jar:na]

at com.aspire.propelstream.odas.topology.bolt.PredictorBolt.makeBeam(PredictorBolt.java:49) ~[stormjar.jar:na]

at com.metamx.tranquility.storm.BeamBolt.prepare(BeamBolt.scala:55) ~[stormjar.jar:na]

at backtype.storm.daemon.executor$fn__5017$fn__5030.invoke(executor.clj:732) ~[storm-core-0.9.3.2.2.4.2-2.jar:0.9.3.2.2.4.2-2]

at backtype.storm.util$async_loop$fn__452.invoke(util.clj:463) ~[storm-core-0.9.3.2.2.4.2-2.jar:0.9.3.2.2.4.2-2]

at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]

at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]

2015-09-02 10:23:52 b.s.d.executor [ERROR]

java.lang.NoSuchMethodError: scala.collection.JavaConverters$.asScalaBufferConverter(Ljava/util/List;)Lscala/collection/convert/Decorators$AsScala;

at com.metamx.tranquility.druid.DruidRollup$.create(DruidRollup.scala:168) ~[stormjar.jar:na]

at com.metamx.tranquility.druid.DruidRollup.create(DruidRollup.scala) ~[stormjar.jar:na]

at com.aspire.propelstream.odas.topology.bolt.PredictorBolt.makeBeam(PredictorBolt.java:49) ~[stormjar.jar:na]

at com.metamx.tranquility.storm.BeamBolt.prepare(BeamBolt.scala:55) ~[stormjar.jar:na]

at backtype.storm.daemon.executor$fn__5017$fn__5030.invoke(executor.clj:732) ~[storm-core-0.9.3.2.2.4.2-2.jar:0.9.3.2.2.4.2-2]

at backtype.storm.util$async_loop$fn__452.invoke(util.clj:463) ~[storm-core-0.9.3.2.2.4.2-2.jar:0.9.3.2.2.4.2-2]

at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]

at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]

I have attached the complete log. Should any external dependency be included for the scala related error? I am using Storm 0.9.3, Tranquility 2.10.

Thank you.

topology-3-1441169561-worker-6701.log (281 KB)

Is anyone else facing this issue??

I added scala 2.11 dependency and I got ClassManifest - NoSuchClassDefError in the Kafka Spout side. Could it be that kafka and Beam Bolt were built using different scala versions?

I have included Tranquility in the maven dependency

io.druid

tranquility_2.10

0.5.0

This is my Beam Factory…
public Beam<Map<String, String>> makeBeam(Map<?, ?> conf, IMetricsContext metrics)

{

    try {

          final CuratorFramework curator = CuratorFrameworkFactory.newClient(

                  "127.0.0.1", new  RetryOneTime(1000));

        curator.start();

    final List<String> dimensions = ImmutableList.of("page","language","user","unpatrolled","newPage","robot","anonymous","namespace","continent","country","region","city");

    final List<AggregatorFactory> aggregators = ImmutableList

        .<AggregatorFactory>of(new CountAggregatorFactory("cnt"));

        final String dataSource = "wikipedia";

        final DruidBeams.Builder<Map<String, String>> builder = DruidBeams.builder(new Timestamper<Map<String, String>>(){

                @Override

                  public DateTime timestamp(Map<String, String> map)

              {
            String timestamp = map.get("timestamp");
            Log.info("in BeamBolt:"+timestamp);
            Log.info("records:"+map.get("page")+","+map.get("language")+","+map.get("user"));

                return new DateTime(timestamp);

              }

            }

          ).curator(curator)
          .discoveryPath("/druid/discovery")
          .location(

            DruidLocation.create(

                "overlord",

                "druid:firehose:%s",

                dataSource

            )

        )

        .rollup(DruidRollup.create(DruidDimensions.specific(dimensions), aggregators, QueryGranularity.NONE))

        .tuning(

            ClusteredBeamTuning.builder()

                               .segmentGranularity(Granularity.DAY)

                               .windowPeriod(new Period("PT1M"))

                               .build()

        );


       

    final Beam<Map<String,String>> beam = builder.buildBeam();

    return beam;

    }
catch (Exception e)
{
    Log.info("caught exception in BeamBolt "+e);

        throw Throwables.propagate(e);

      }

}

Hey Sai,

This does look like a Scala version conflict. You should be able to get things to work by using the Scala 2.10 builds of both Tranquility and Kafka (both projects publish builds for that version). Mixing 2.10 and 2.11 builds will unfortunately not work.

Gian

Hi Gian,

Thanks for the reply. When I used the eariler version(0.3.5) or when I try to build 0.5 manually using scala 2.10 I get the following error

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile) on project topology: Compilation failure: Compilation failure:
[ERROR] /root/Bolt.java:[7,38] package com.fasterxml.jackson.databind does not exist
[ERROR] /root/Bolt.java:[10,25] package com.metamx.common does not exist
[ERROR] /root/Bolt.java:[21,28] package io.druid.granularity does not exist
[ERROR] /root/Bolt.java:[22,34] package io.druid.query.aggregation does not exist
[ERROR] /root/Bolt.java:[23,34] package io.druid.query.aggregation does not exist
[ERROR] /root/Topology.java:[65,16] cannot access com.metamx.common.scala.Logging

There’s an official 2.10 build that you can use: http://search.maven.org/#artifactdetails%7Cio.druid%7Ctranquility_2.10%7C0.5.0%7Cjar

You can get it by including this in your pom.xml, if you’re using Maven:

io.druid

tranquility_2.10

0.5.0

I used the same build in the begining, and hence this thread :slight_smile:

Infact, after using the Kafka_2.11 build, the above mentioned error has gone, instead I get
2015-09-04 10:29:24 b.s.util [ERROR] Async loop died!
java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class
at com.twitter.finagle.Stack$Params$Prms.(Stack.scala:263) ~[stormjar.jar:na]
at com.twitter.finagle.Stack$Params$.(Stack.scala:286) ~[stormjar.jar:na]
at com.twitter.finagle.Stack$Params$.(Stack.scala) ~[stormjar.jar:na]
at com.twitter.finagle.builder.ClientConfig$.(ClientBuilder.scala:109) ~[stormjar.jar:na]
at com.twitter.finagle.builder.ClientConfig$.(ClientBuilder.scala) ~[stormjar.jar:na]
at com.twitter.finagle.builder.ClientBuilder.(ClientBuilder.scala:237) ~[stormjar.jar:na]
at com.twitter.finagle.builder.ClientBuilder$.apply(ClientBuilder.scala:35) ~[stormjar.jar:na]
at com.metamx.tranquility.finagle.FinagleRegistry.mkclient(FinagleRegistry.scala:47) ~[stormjar.jar:na]
at com.metamx.tranquility.finagle.FinagleRegistry.checkout(FinagleRegistry.scala:89) ~[stormjar.jar:na]
at com.metamx.tranquility.druid.IndexService.(IndexService.scala:50) ~[stormjar.jar:na]
at com.metamx.tranquility.druid.DruidBeams$Builder.buildBeam(DruidBeams.scala:158) ~[stormjar.jar:na]
at com.aspire.propelstream.odas.topology.bolt.PredictorBolt.makeBeam(Bolt.java:80) ~[stormjar.jar:na]
at com.metamx.tranquility.storm.BeamBolt.prepare(BeamBolt.scala:55) ~[stormjar.jar:na]

Hmm, this really does seem like a scala version conflict. Are you using 2.10 versions of all scala-related artifacts? If you do a “mvn dependency:list” or “mvn dependency:tree” do you see anything using any other version of scala?

Hi Gian,

After moving tranquility higher up the order and adding kafka-2.10 explicitly, the issue got resolved.

Thanks for your help :slight_smile: