How to get metrics messages sent by HttpPostEmitter in the doPost method of Http server?

Hi Experts,

I use HttpPostEmitter to send the metrics msg to a Http Server which is wrote with Jetty Java API by myself, and this server doPost method really could accept request sent by Druid, however I could anybody show how to get the metrics json data in the doPost method?

Below is my Jetty http server code and http servlet code for your information. Please help show me the correct code and let me know if I misunderstood anything. Thanks!

Jetty http server code
public class JettyServletExampleServer {
private static final Logger LOGGER = LoggerFactory
.getLogger(JettyServletExampleServer.class);

public static void main(String[] args) {
    try {
        Server server = new Server(8080);

        ServletContextHandler context = new ServletContextHandler(
                ServletContextHandler.SESSIONS);
        context.setContextPath("/");
        server.setHandler(context);
        context.addServlet(new ServletHolder(new DruidServlet()), "/druid");
        LOGGER.info("server start.");
        server.start();
        server.join();
    } catch (Exception e) {
        e.printStackTrace();
    }

}

}

http servlet code
public class DruidServlet extends HttpServlet {
private static final Logger LOGGER = LoggerFactory
.getLogger(DruidServlet.class);

private static final long serialVersionUID = -4012838481920999524L;

@Override
public void doPost(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {
    LOGGER.info("welcome to my server. It's a POST request.");
    String query = request.getParameter("druid.emitter.http");     // query is always null

    LOGGER.info(String.format("receive query: %s", query));

    // TODO: Forward the Json message to somewhere
}

}

Hi Qiyun, can you verify that HTTP requests are actually being sent? As in, there are non-empty requests? Can you post your configuration for the HTTP emitter? Can you also post the body of the request message? I don’t recall if the correct parameter is druid.emitter.http.

Hi Fangjin,

Yes, our broker service could send metrics via http, and finally we use druid-metrics-to-kafka project to forward these http messages to Kafka, and now we could consume the regarding Kafka topic using a Druid datasource. Below is our conf in config/broker/runtime.properties:
# Enable Broker Node monitoring
druid.monitoring.monitors=[“io.druid.client.cache.CacheMonitor”]
druid.monitoring.emissionPeriod=PT1M
druid.emitter=http
druid.emitter.http.timeOut=PT10M
druid.emitter.http.flushMillis=60000
druid.emitter.http.flushCount=100
druid.emitter.http.recipientBaseUrl=http://gateway.host:9999/metrics

Above solution works well for us now. However, we are just wondering how to get the metrics in the JettyServletExampleServer? For this solution, our conf is:
druid.emitter=http
druid.emitter.http.timeOut=PT10M
druid.emitter.http.flushMillis=60000
druid.emitter.http.flushCount=100
druid.emitter.http.recipientBaseUrl=http://127.0.0.1:8080/druid

And in the broker logs, seems it already successfully sent out the metrics via http:
2015-11-25 03:58:18,873 INFO c.m.h.c.p.ChannelResourceFactory [HttpPostEmitter-1-0] Generating: http://127.0.0.1:8080

At the same time, the JettyServletExampleServer also could get some http post request and then print some info like:
2015/11/25 12:00:18.896 INFO welcome to my server. It’s a POST request.
2015/11/25 12:00:18.897 INFO receive query: null

We could mention that ‘receive query: null’ and that means we failed to extract the real metrics info from the http post request. So the question is how to modify our JettyServletExampleServer?

Thanks!

在 2015年11月16日星期一 UTC+8上午12:41:51,Fangjin Yang写道:

Hi Qiyun, you can try the development lists, but I think given the fact that the public lists are a volunteer of people’s time, it will be hard to find someone to go over your code. If you need dedicated help with Druid, http://imply.io/ is around.