Elasticsearch logs are generated in the Logserver/elasticsearch-1.5.2/log directory. If the log files are not moved or deleted, disk space can fill up. Can Elasticsearch log files be purged automatically?
Resolution
Update the Elasticsearch log configuration, and use the MaxBackupIndex option to determine how many backup files are kept before the oldest is erased.
The Elasticsearch logs configuration can be found in Logserver/elasticsearch-1.5.2/config/logging.yml.
The default configuration is:
"
file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
"
With this configuration a file will be created every day in order to save the previous day's log. As a result, the number of log files will increase, and it can lead to the disk full problem.
To restrict the amount of backup files, set MaxBackupIndex:
-
Stop these Talend services:
Talend Log Server search engine
Talend Log Server analytics and visualization platform
- Go to the <Talend_logserv_installation>elasticsearch-2.4.0\config\ folder.
- Open the logging.yml file and modify it as follows, adding maxFileSize and maxBackupIndex:
"
file:
type: rollingFile
file: ${path.logs}/${cluster.name}.log
maxFileSize : 200KB
maxBackupIndex: 4
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
"
- Save the file and restart the services.
Related Content:
Qlik Talend Product: Reducing the logging threshold for Elasticsearch in Talend Log Server
Environment
Talend Data Integration
