topic Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine. in Talend Studio

How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

nvtd4000 — Fri, 15 Nov 2024 23:46:16 GMT

I am using Talend Studio 7.3.1 and Remote Engine 2.10.6 on DEV server. I have a very simple job and in project settings, i modified log4j2.xml to log information to a seperate files. When i run the job in Talend Studio, I can see information in log files that i specified in log4j2.xml

Then I deploy the job to Remote Engine. So far so good. Job is running okay. But there is no log file created (log files that i want to log information to). I goto TalendRemoteEngine/data/dsrunner/exec/microservices/7ad99da8-68c5-438e-a840-a8da8eee9eae/rest1/config and see a brand new log4j2.xml with all default values (rest1 in the path is my job name). It looks like my project settings log4j2.xml only belongs to Studio in local. When we deploy job to Remote Engine, It doesn't go with the job to Remote engine.

Do you know how to make that happens: make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

nvtd4000 — Tue, 07 Sep 2021 03:25:32 GMT

Thanks for the reply. But you haven't answer my question: How to do that?

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

Anonymous — Tue, 07 Sep 2021 09:25:12 GMT

Hello,

Please have a look at this article about: https://community.talend.com/s/article/Do-you-want-to-write-the-logs-in-a-specific-way-in-TMC-when-run-against-the-Remote-Engine-j5JHp

Let us know if it is what you are looking for.

Best regards

Sabrina

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

nvtd4000 — Tue, 07 Sep 2021 15:00:56 GMT

The Link youshare is invalid. Could you please give me the link again

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

Anonymous — Wed, 08 Sep 2021 02:47:15 GMT

Hello,

It is not invalid. This is an internal KB article.

I pasted the content here and uploaded related zip files.

Problem Description

Do you have a requirement to compose the log, in so and so way in Studio? Do you want to have the same setup, that it writes the log to the Remote Engine also the same way when executed from the TMC?

Solution

1) Attached job test_test.zip used in our testing.

2) Attached log4j2_modified.xml

3) In the job tJava having below snippet log4j api .

4) JVM arguments having -Dlog4j.configurationFile= file:///C:/tmp/log4j2_modified.xml (file:///C:/tmp/log4j2.xml)

When deployed and run from TMC we can get the desired log in above log4j.configurationFile mentioned location and output is as below,

Starting job Copy_of_test_test at 16:20 30/04/2020.

[INFO ] 2020-04-30 16:20:49,643 Copy_of_test_test.runJobInTOS() - TalendJob: 'Copy_of_test_test' - Start.

[statistics] connecting to socket on port 3837

[statistics] connected

[DEBUG] 2020-04-30 16:20:49,691 Copy_of_test_test.tDBInput_1Process() - tLogRow_1 - Start to work.

[DEBUG] 2020-04-30 16:20:49,693 Copy_of_test_test.tDBInput_1Process() - tDBInput_1 - Start to work.

[DEBUG] 2020-04-30 16:20:49,695 Copy_of_test_test.limitLog4jByte() - tDBInput_1 - Parameters:DB_VERSION = MYSQL_5 | USE_EXISTING_CONNECTION = false | HOST = "localhost" | PORT = "3306" | DBNAME = "log_test" | USER = "root" | PASS = enc:... | TABLE = "backup" | QUERYSTORE = "" | QUERY = "SELECT * FROM `backup`" | SPECIFY_DATASOURCE_ALIAS = false | PROPERTIES = "noDatetimeStringSync=true" | ENABLE_STREAM = false | TRIM_ALL_COLUMN = false | TRIM_COLUMN = [{TRIM=false, SCHEMA_COLUMN=newColumn}] | UNIFIED_COMPONENTS = tMysqlInput |

[DEBUG] 2020-04-30 16:20:51,634 Copy_of_test_test.tDBInput_1Process() - tDBInput_1 - Driver ClassName: com.mysql.jdbc.Driver.

[DEBUG] 2020-04-30 16:20:51,635 Copy_of_test_test.tDBInput_1Process() - tDBInput_1 - Connection attempt to 'jdbc:mysql://localhost:3306/log_test?noDatetimeStringSync=true' with the username 'root'.

[DEBUG] 2020-04-30 16:20:52,017 Copy_of_test_test.tDBInput_1Process() - tDBInput_1 - Connection to 'jdbc:mysql://localhost:3306/log_test?noDatetimeStringSync=true' has succeeded.

[DEBUG] 2020-04-30 16:20:52,018 Copy_of_test_test.tDBInput_1Process() - tDBInput_1 - Executing the query: 'SELECT *

FROM `backup`'.

[DEBUG] 2020-04-30 16:20:52,027 Copy_of_test_test.tDBInput_1Process() - tDBInput_1 - Retrieving records from the database.

[DEBUG] 2020-04-30 16:20:52,028 Copy_of_test_test.tDBInput_1Process() - tDBInput_1 - Closing the connection to the database.

[DEBUG] 2020-04-30 16:20:52,029 Copy_of_test_test.tDBInput_1Process() - tDBInput_1 - Connection to the database closed.

[DEBUG] 2020-04-30 16:20:52,030 Copy_of_test_test.tDBInput_1Process() - tDBInput_1 - Retrieved records count: 0 .

[DEBUG] 2020-04-30 16:20:52,030 Copy_of_test_test.tDBInput_1Process() - tDBInput_1 - Done.

[INFO ] 2020-04-30 16:20:52,030 Copy_of_test_test.tDBInput_1Process() - tLogRow_1 - Printed row count: 0.

[DEBUG] 2020-04-30 16:20:52,030 Copy_of_test_test.tDBInput_1Process() - tLogRow_1 - Done.

[statistics] disconnected

[INFO ] 2020-04-30 16:20:52,032 Copy_of_test_test.main() - TalendJob: 'Copy_of_test_test' - Done.

Job Copy_of_test_test ended at 16:20 30/04/2020. [Exit code = 0]

Let us know if it helps.

Best regards

Sabrina

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

Anonymous — Wed, 08 Sep 2021 02:47:43 GMT

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

nvtd4000 — Wed, 08 Sep 2021 04:17:22 GMT

Thanks. The log4j.xml is a html file:

<head>

<title>Submit Form</title>

</head>

......

test_test.zip is correupted and I am not able to import it to talend studio.

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

Anonymous — Wed, 08 Sep 2021 07:24:01 GMT

Hello,

It seems the zip file is corrupted. We will check it with the article author to see if there is any valid source for it.

In addition to that, you can enable log4j and log4j2 logging configured in Talend Studio for tasks running on Remote Engines.

Here is online user guide about: TalendHelpCenter: Enabling custom logging

Best regards

Sabrina

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

DBOUAKKAZ — Mon, 13 Sep 2021 12:37:50 GMT

Hello,

I'm encountering exactly the same issue. Did you find a solution?

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

Anonymous — Tue, 14 Sep 2021 02:59:49 GMT

Hello,

So far, we are not able to find this test_test file.

You can enable log4j and log4j2 logging configured in Talend Studio for tasks running on Remote Engines.

Here is online user guide about: TalendHelpCenter: Enabling custom logging

Best regards

Sabrina

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

DBOUAKKAZ — Tue, 14 Sep 2021 08:53:45 GMT

Hello and thank you for your answer, but I've already tried setting the "job.log.config.embedded" parameter value to true and nothing happened (even the logs on the TMC are still there)

Is it possible to just have the snippet log4j api in the tjava that he's referring to?

Re: How to make log4j2.xml in project settings in Studio goes with the job to Remote Engine.

Anonymous — Mon, 27 Dec 2021 21:14:23 GMT

Hello,

I noticed that the ESB tag is used and you are looking under the microservices folder for your Remote Engine. Custom log4j is configured differently than standard jobs. Please review the documentation below and let me know if you are able to get your logging working:

https://help.talend.com/r/en-US/Cloud/remote-engine-user-guide-linux/collecting-microservice-logs

Best Regards,

Chris