Separate log file for each node

Hi,

I wanted to be able to store the emitted logs of each node (coordinator, broker, etc.) in separate log files. How would I go about doing that? I think a similar question was asked here (https://groups.google.com/forum/#!searchin/druid-user/log$20file/druid-user/J1NLr9eHwoU/ekpfbq1zBwAJ) but I don’t think it was ever resolved.

Thanks a lot

Thomas

Hey Thomas,

One option is to have a different log4j2.xml for each node (perhaps put this in a per-node configuration directory) and have each node write to a different file appender.

Another option is to have log4j2 log to stdout, and use a service supervisor to control where the logs go. runit + svlogd (http://smarden.org/runit/) works pretty well for this.

Thomas -

Can you also try the following -

Create a new appender in your log4j2.xml as follows -

%d %p %c{1.} [%t] %m%n

``

observe how I made the log name derived from runtime arguments -
“${sys:logfilename}.log”

``

And now when you start your nodes append a file name as a jvm argument. For example for history node

java -Xmx1024m -Dlogfilename=history -XX:MaxDirectMemorySize=4294967296 -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath config/_common:config/historical:lib/* io.druid.cli.Main server historical &

``

This will create history.log

Hope that helps.

This doesn’t work, it just creates ${sys/logFilename} file

Did you specify the -Dlogfilename=history in your java command like I mentioned above?

Yes, it works. Thank you.

Hi ,

I tried the above method on latest version of Druid (0.9.2) and it works. However unable to see similar behaviour in an older version od Druid 0.8.3.

Is this expected or is there anything I am missing.

Got this fixed. The config directory with log4j2.xml was supposed to be put before the lib jars in the classpath. The following classpath order works.

java -cp config/druid/_common:config/historical:lib/*

Regards

Rajitha

Thanks, it works perfectly