Confirming that Tranquility-Kafka is running and processing messages

I’m in the process of replacing a Realtime configuration instance with Tranquility-Kafka. I’d like to run the two side by side while I verify that Tranquility is functioning correctly. I have two questions concerning this verification process:

  1. Does Tranquility have its own logs? I’ve checked the /var/log/druid directory (where the other Druid services logging) but haven’t seen anything referring to Tranquility (I even checked the realtime.log).

  2. Does Tranquility have any HTTP routes that I can query to confirm that everything is functional?

  1. Tranquility-Kafka generates its own logs separately from Druid since it runs as a separate process. The default is to write to stdout which you could redirect to a file. Alternatively, you can supply a custom logger configuration to output directly to a file (see http://logback.qos.ch/manual/configuration.html)

  2. No, Tranquility-Kafka doesn’t have HTTP status routes, but does generate periodic log events showing received/sent/dropped/unparseable counts.

If you haven’t already looked into the Kafka indexing service, I recommend you give the docs a look to see if it makes sense for you: http://druid.io/docs/0.9.1.1/development/extensions-core/kafka-ingestion.html. The main advantages of the indexing service over tranquility-kafka are 1) runs within the standard Druid services and doesn’t require an additional process, 2) has guarantees for exactly-once ingestion, and 3) can ingest arbitrarily timestamped data. As a bonus, it does have an HTTP route you can query for status.