Tranquility fails to start with json flatten spec.

Hi All,

Can anyone confirm still below issue persists in Druid N tranquility server??

Because I am facing same issue .

Hi All.

I am using Tranquility to push real-time streams and for simple multi-line JSON it is working as expected. I need to push multi-line **nested **JSON objects and I have implemented using JSON Flatten Spec as in this example. Tranquility starts fine with just path fields however, it fails to start when root type field is specified. Stacktrace below.
java.lang.IllegalArgumentException: Instantiation of [simple type, class io.druid.data.input.impl.StringInputRowParser] value failed: **json can not be null or empty**
at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2774)
at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2700)
at io.druid.segment.indexing.DataSchema.getParser(DataSchema.java:101)
at com.metamx.tranquility.druid.DruidBeams$.fromConfigInternal(DruidBeams.scala:293)
at com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:199)
at com.metamx.tranquility.server.http.ServerMain$$anonfun$2.apply(ServerMain.scala:118)
at com.metamx.tranquility.server.http.ServerMain$$anonfun$2.apply(ServerMain.scala:98)
at com.metamx.common.scala.collection.package$MapLikeOps$$anonfun$strictMapValues$1.apply(package.scala:143)
at com.metamx.common.scala.collection.package$MapLikeOps$$anonfun$strictMapValues$1.apply(package.scala:143)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at scala.collection.immutable.Map$Map1.foreach(Map.scala:116)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at com.metamx.common.scala.collection.package$MapLikeOps.strictMapValues(package.scala:143)
at com.metamx.tranquility.server.http.ServerMain$.createServlet(ServerMain.scala:98)
at com.metamx.tranquility.server.http.ServerMain$.main(ServerMain.scala:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.twitter.app.App$$anonfun$nonExitingMain$3.apply(App.scala:168)
at com.twitter.app.App$$anonfun$nonExitingMain$3.apply(App.scala:167)
at scala.Option.foreach(Option.scala:257)
at com.twitter.app.App$class.nonExitingMain(App.scala:167)
at com.metamx.tranquility.server.http.ServerMain$.nonExitingMain(ServerMain.scala:49)
at com.twitter.app.App$class.main(App.scala:133)
at com.metamx.tranquility.server.http.ServerMain$.main(ServerMain.scala:49)
at com.metamx.tranquility.distribution.DistributionMain$.main(DistributionMain.scala:34)
at com.metamx.tranquility.distribution.DistributionMain.main(DistributionMain.scala)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Instantiation of [simple type, class io.druid.data.input.impl.StringInputRowParser] value failed: json can not be null or empty
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapException(StdValueInstantiator.java:405)
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:234)
at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:167)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:398)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84)
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132)
at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:41)
at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2769)
... 29 more
Caused by: java.lang.IllegalArgumentException: json can not be null or empty
at com.jayway.jsonpath.internal.Utils.notEmpty(Utils.java:383)
at com.jayway.jsonpath.JsonPath.compile(JsonPath.java:465)
at com.metamx.common.parsers.JSONPathParser.generateFieldPaths(JSONPathParser.java:139)
at com.metamx.common.parsers.JSONPathParser.<init>(JSONPathParser.java:64)

regards,

Sumeet Lalvani

732-430-9586

Hi Sumeet,

Are you getting the same stack trace as the issue you referenced? It sounds like a malformed JSON flatten spec. I would double-check this section of the configuration. If you post it, some on the user group may be able to help you correct the issue.

Also to be clear - Druid doesn’t support multi-line JSON, but rather uses newline delimited JSON (meaning that each JSON object must be written on a single line, and a newline character is used to separate objects).

{ “created_at”: “Thu Jun 27 22:53:37 +0000 2019”, “id”: 1144378277940879400, “id_str”: “1144378277940879360”, “text”: “jhon is good”, “truncated”: true, “entities”: { “hashtags”: , “symbols”: , “user_mentions”: [ { “screen_name”: “MsPackyetti”, “name”: “Brittany Packnett”, “id”: 239509917, “id_str”: “239509917”, “indices”: [ 0, 12 ] }, { “screen_name”: “Dell”, “name”: “Dell”, “id”: 58561993, “id_str”: “58561993”, “indices”: [ 13, 18 ] }, { “screen_name”: “BoozAllen”, “name”: “Booz Allen Hamilton”, “id”: 17375116, “id_str”: “17375116”, “indices”: [ 19, 29 ] }, { “screen_name”: “somone”, “name”: “U.S. Bank”, “id”: 15577098, “id_str”: “15577098”, “indices”: [ 30, 37 ] }, { “screen_name”: “Dell”, “name”: “Dell”, “id”: 58561993, “id_str”: “58561993”, “indices”: [ 41, 46 ] } ], “urls”: [ { “url”: “https://t.co/saRaIA5Znw”, “expanded_url”: “https://twitter.com/i/web/status/1144378277940879360”, “display_url”: “twitter.com/i/web/status/1…”, “indices”: [ 117, 140 ] } ] }, “metadata”: { “iso_language_code”: “en”, “result_type”: “recent” }, “source”: “<a href=”[http://twitter.com](http://twitter.com%5C)" rel=“nofollow”>Twitter Web Client", “in_reply_to_status_id”: 1143857245496709100, “in_reply_to_status_id_str”: “1143857245496709120”, “in_reply_to_user_id”: 239509917, “in_reply_to_user_id_str”: “239509917”, “in_reply_to_screen_name”: “MsPackyetti”, “user”: { “id”: 4104671473, “id_str”: “4104671473”, “name”: “ʇuǝɯǝlǝ ıɥʞ”, “screen_name”: “Khi_Element”, “location”: “Earth”, “description”: “Your untypical Holly Golightly-like character. Lover of all living things. Food junkie. Book addict. Caffeine-fueled social critic.”, “url”: null, “entities”: { “description”: { “urls”: } }, “protected”: false, “followers_count”: 46, “friends_count”: 812, “listed_count”: 0, “created_at”: “Mon Nov 02 18:42:44 +0000 2015”, “favourites_count”: 1292, “utc_offset”: null, “time_zone”: null, “geo_enabled”: false, “verified”: false, “statuses_count”: 947, “lang”: “en”, “contributors_enabled”: false, “is_translator”: false, “is_translation_enabled”: false, “profile_background_color”: “000000”, “profile_background_image_url”: “http://abs.twimg.com/images/themes/theme1/bg.png”, “profile_background_image_url_https”: “https://abs.twimg.com/images/themes/theme1/bg.png”, “profile_background_tile”: false, “profile_image_url”: “http://pbs.twimg.com/profile_images/733406985228419073/jqpYwSXd_normal.jpg”, “profile_image_url_https”: “https://pbs.twimg.com/profile_images/733406985228419073/jqpYwSXd_normal.jpg”, “profile_banner_url”: “https://pbs.twimg.com/profile_banners/4104671473/1463692839”, “profile_link_color”: “9266CC”, “profile_sidebar_border_color”: “000000”, “profile_sidebar_fill_color”: “000000”, “profile_text_color”: “000000”, “profile_use_background_image”: true, “has_extended_profile”: false, “default_profile”: false, “default_profile_image”: false, “following”: null, “follow_request_sent”: null, “notifications”: null, “translator_type”: “none” }, “geo”: null, “coordinates”: null, “place”: null, “contributors”: null, “is_quote_status”: false, “retweet_count”: 0, “favorite_count”: 0, “favorited”: false, “retweeted”: false, “lang”: “en” }

twitter-search-spec.json (2.08 KB)

Hi Sumeet,

Try setting a value for ‘expr’, so instead of:

{
“type”: “root”,
“name”: “text”
}

try:

{
“type”: “root”,
“name”: “text”,
“expr”: “text”
}