topic Re: data and log folder in elasticsearch installation in Talend Studio

data and log folder in elasticsearch installation

Anonymous — Sat, 16 Nov 2024 10:22:16 GMT

Hi All
I have a customer whose logs and data directory in elastic search installation is getting very large.
In his installation:
logs folder in elasticsearch installation in logserver is apprx 13 GB and data folder is similar size.
Can somebody please explain what these files/folders are for and how we can limit how much space they take or maybe i can ask customer to move those folder or delete them if it is OK to do so.

Thanks in advance
tmishra

Re: data and log folder in elasticsearch installation

_AnonymousUser — Fri, 23 Sep 2016 13:51:56 GMT

This issue is due to the config of the logging.yml file as the logging is done in log4j.
There's an issue with how the customer has their logging.yml file set up. For example, the default logging.yml file has this code for the file appender:

  file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: " %m%n"

It sounds like the files aren't being rolled over daily and therefore is creating one hige ${cluster_.name}.log file.

Re: data and log folder in elasticsearch installation

Anonymous — Sun, 25 Sep 2016 14:41:57 GMT

Dear Tinker,
Thanks for your reply. Can you tell me where these two parameters are set? $( path.logs} and ${cluster.name} ?

Thanks
Kind Regards
tmishra

Re: data and log folder in elasticsearch installation

_AnonymousUser — Mon, 26 Sep 2016 09:13:28 GMT

Dear Tinker,
Thanks for your reply. Can you tell me where these two parameters are set? $(path.logs} and ${cluster.name} ?

Thanks
Kind Regards
tmishra

You can find these in your elasticsearch config folder (mine is elasticsearch-1.5.2/config) in elasticsearch.yml