Help on Log4j setting for different LogStash servers

[Sanjay5]

Hi All,
I need help on setting Log4j setting for Different Logstash servers.
Scenario:
I have 3 different Logstash servers, and 3 different Job servers.
Logstash server is different than Job Server.
I have added Logstash appender in Log4j setting of my Studio. (See below screen shot)

Issue: 
I have set log4j in my studio only for 1 logstash server not for all 3.
I am running the job in 3 different job servers. 
I don't want to change the setting for each of the servers (DEV, QA and PROD)
If i set up 3 logstash apenders for each logstash servers, logs will be going to (DEV, QA and PROD) together, which i dont want.

Question:
Is there any way to setup LOG4J so that i get the logs when i run the job in any of the job servers.
I want the DEV logs, when i run the job in DEV
QA logs when i run the job in QA, like wise for PROD. 

Note: 
Logging works fine if i setup 1 Logstash appender in my LOG4J setting.
I can't use "Localhost" as Logstash and Job servers are different.
Please let me know if you have any idea on this.

[vapukov]

first of all - screenshot not attached
but in any case - You can define context variable for any settings, and create 3 context group - 1 per Job server
depending from server You choose proper context group for Job

[Sanjay5]

Hi,
Thanks for the reply,
Please find below screen shot


I tried using context variable, but if i provide context variable as below, It is not working in LOG4J setting.
<!-- logstash appender -->
    <appender name="LOGSTASH" class="org.apache.log4j.net.SocketAppender">
        <param name="RemoteHost" value="context.logstash_hostname"/>
        <param name="Port" value="8052"/>
    </appender>

[vapukov]

try change:

<!-- logstash appender -->
    <appender name="LOGSTASH" class="org.apache.log4j.net.SocketAppender">
        <param name="RemoteHost" value="context.logstash_hostname"/>
        <param name="Port" value="8052"/>
</appender>

to 

<!-- logstash appender -->
    <appender name="LOGSTASH" class="org.apache.log4j.net.SocketAppender">
        <param name="RemoteHost" value=context.logstash_hostname/>
        <param name="Port" value="8052"/>
</appender>

I have only 1 log stash at the moment, so it hard to test, but at least it not return errors

[Sanjay5]

I tried appender using context variable as below:
<!-- logstash appender -->
    <appender name="LOGSTASH" class="org.apache.log4j.net.SocketAppender">
        <param name="RemoteHost" value=context.logstash_hostname/>
        <param name="Port" value="8052"/>
</appender>

It is not accepting context variable in Studio log4j setting.
It is not giving error but logs are not getting logged.

[Sanjay5]

Few things i wanted to confirm:
1. Are you able to get component level logs using context variable in log4j setting.
2. I am giving logstash appender in Studio log4j setting (See attached), SO is this the only way to get component level logs or is there any other way i can add logstash appender to get component level logs.

[Sanjay5]

I got the solution for this issue:
For each of the 3 job servers, we need to include Logserver IP address in the "hosts file" located at "/ect/hosts"
What i did is as below

Added  logstash IP address  in Job Server 1 "hosts file"
XX.XX.XXX.XXX    talend_logstash.aexp.com


Added  logstash IP address  in Job Server 2 "hosts file"
YY.YY.YYY.YYY    talend_logstash.aexp.com


Added  logstash IP address  in Job Server 3 "hosts file"
ZZ.ZZ.ZZZ.ZZZ    talend_logstash.aexp.com

Now i am using below details in logstash appender:
<!-- logstash appender -->
    <appender name="LOGSTASH" class="org.apache.log4j.net.SocketAppender">
        <param name="RemoteHost" value="talend_logstash.aexp.com"/>
        <param name="Port" value="8052"/>
    </appender>