How to Integrate Druid with C code application

Hi

Thanks for the info.

My C application itself will generate logs/data with time stamps periodically. My C application would like to write these logs into Kafka bus, which are to be consumed and stored in Druid Real time node . I will analyze these stored logs based on time stamp/data at later point of time using my own queries.

  1. I have downloaded the druid and started zookeeper, kafka and druid nodes using the scripts mentioned in the 'Quick Startup Guide" on my linux machine.

I could send data using ‘kafka-producer’ script in the kafka directory and received them on the Realtime node.

  1. Now I would like to integrate the druid with my C application. Please let me know what are the options available? Do i need to link all the .jar files in the druid/lib directory? or any C library available for Druid like “librdkafa” for Kafka?

Thanks and regards

Mahesh G

Mahesh G <gmahesh.tsdb@gmail.com >: Jul 13 06:18PM +0530

    Hi

I am planning to integrate Druid with my C code application and build a

single executable. Could you please let me know how to integrate Druid with

any application written in C code.

My plan is to launch the zookeeper, kafka bus and druid nodes (Coordinator,

Historical and Real time nodes) from my C application and my C application

will send data to Kafka bus which is to be consumed by Druid Real time node.

Thanks and regards

Mahesh Gullipalli

      "charles.allen" <charles.allen@metamarkets.com        >: Jul 13 09:28AM -0700

    Are you trying to get data into your c program? If that's all you're trying

to do then you can use the standard REST endpoints. Both the R

<https://github.com/druid-io/RDruid> and Python

<https://github.com/druid-io/pydruid> supplied frameworks do this. And I

actually suggest you consider if your project is better fit in Python or R

unless you really really need C for some reason.

Hi,
if you are trying to analyze your metric events produced by your C application, you don’t need any coupling between druid and your application.

All you need it to send the data from your application to the kafka broker. This means you have to couple your application with a kafka producer like (https://github.com/edenhill/librdkafka)

Once that done you can use the druid cluster to ingest that data.

So there is no reason to embed druid into your application.

I hope that helps.

There is duplication between this thread and https://groups.google.com/forum/#!topic/druid-development/pSsUbq9hrBM. Please choose and thread and keep discussions in one thread please.