[druid-user] How to set druid's logging properly

Hi Druid,

Druid version:0.22.1

“apache-druid-0.22.1/conf/druid/cluster/_common/log4j2.xml” and “common.runtime.properties” has been configed, but the log file exists in both directory, “apache-druid-0.22.1/logs” and “apache-druid-0.22.1/var/sv”

How to set logging properly? Objective:The log file can be divided into several small files, and the history file can be deleted automatically. Thank you!

MiddleManagerConfig:

1、apache-druid-0.22.1/conf/druid/cluster/_common/log4j2.xml

/usr/local/apache-druid-0.22.1/logs coordinator-overlord broker router historical middleManager




























2、apache-druid-0.22.1/conf/druid/cluster/_common/common.runtime.properties

Extensions
druid.extensions.loadList=[“druid-datasketches”, “druid-lookups-cached-global”,“mysql-metadata-storage”,“druid-hdfs-storage”,“druid-histogram”,“druid-kafka-indexing-service”]

Logging

druid.startup.logging.logProperties=true

Zookeeper

druid.zk.service.host=hd1:2181,hd2:2181,hd3:2181
druid.zk.paths.base=/druid

For MySQL

druid.metadata.storage.type=mysql
druid.metadata.storage.connector.connectURI=jdbc:mysql://mysqlnode:3306/druid?characterEncoding=UTF-8
druid.metadata.storage.connector.user=druid
druid.metadata.storage.connector.password=druid

Deep storage - HDFS

druid.storage.type=hdfs
druid.storage.storageDirectory=hdfs://myha01/druid/segments

Indexing service logs

druid.indexer.logs.type=hdfs
druid.indexer.logs.directory=hdfs://myha01/druid/indexing-logs
#druid.indexer.task.restoreTasksOnRestart=true
#druid.indexer.task.gracefulShutdownTimeout=PT1M

Monitoring

druid.monitoring.monitors=[“org.apache.druid.java.util.metrics.JvmMonitor”]
druid.emitter=logging
druid.emitter.logging.logLevel=trace

Storage type of double columns

druid.indexing.doubleStorage=double

druid.host=dd2

Hi,

You can set a Log Retention Policy to enable deletion of old task logs.

For my own clarification, may I ask what you mean by dividing the logs into several small files?

Best,

Mark