How do I get job specific logs? (Beginner)

[Anonymous]

Hello everyone,

I am a beginner in Talend, I currently work on Talend Open Studio for Data Integration.

I need to get all informations from my job / subjob and insert them into a table:

JOB_NAME character # Job Name
LOAD_TYPE character # Loading status (Insert / Update)
STATUS character # Job status / under job (OK / KO)
JOB_STIME date # Start date of job / subjob
JOB_ETIME date # End date of job / subjob

I managed to get the JOB_NAME thanks to a tMap connected to the tLogCatcher (field : job) but I do not know how to get the other fields despite my research.

In thanking you and apologizing for this trivial question for Talend experts.

Regards

[Anonymous · Accepted solution]

This tutorial on the Talend AMC might help you.....

https://www.rilhia.com/tutorials/talend-activity-monitoring-console-amc

[Anonymous]

Hello rhall_2_0, 

Thanks for your help but I really don't understand your solution. 

I need something in my job like in tMap ou tLogCatcher to log my data no ? 

[Anonymous]

Did you look at the linked tutorials? This one shows you how to switch the AMC on per job (https://www.rilhia.com/tutorials/switch-activity-management-console-amc-functionality-talend-job). Essentially this Talend functionality just needs to be switched on and it will carry out a low level of logging for you automatically. The tutorial I sent you talks about the database tables you need to set up and the tutorial I have posted here shows how to switch it on for your job. Once you have done both, you can see all of the stats generated for your job without having to manually code anything

[Anonymous]

Ok but I already do these tutorials and succeed to have basic infos. 

But I need these particular infos : 

LOAD_TYPE character # Loading status (Insert / Update)
STATUS character # Job status / under job (OK / KO)
JOB_STIME date # Start date of job / subjob
JOB_ETIME date # End date of job / subjob

How should I do it ? Thank you 

[Anonymous]

The only data you have given as an example that cannot be gained from the AMC is the LOAD TYPE. The rest of the information is available. To get this you have several options. This will help you extend the AMC.

The AMC uses process ids to log your results (PID). In Talend you have access to the PID in Java code. If you use  a tJava component you can use the code below to print your PID to the output window......

System.out.println("My pid is "+pid);

What you can do is add a tPostJob component to your job and attach some components to provide extra logging using this PID. You will need to create your own database table for this. Now if you make the pid a key in your database table, you can link whatever extra information to your AMC logging you want. So you can add your LOAD TYPE data. So long as it is linked to the PID, you can get all of the other standard AMC data really easily.

[Anonymous]

Ok well, I have succeeded to have my infos but I don't understand how can I get the LOAD_TYPE. 

I understand your explanation (tPostJob component, creation of db ...) 

But where the Update / Insert status is stored ? 

I would like to have the number of update / insert lines too ? 

Thank you for your help. Regards

[Anonymous]

Look at the Global Variables section of this document (https://help.talend.com/reader/hfF27SBEP9Avc0ofq7IDjw/E6MTIua23KeeVBOs_R0iOA) as an example of how you can use database components (and other components reading and writing data). 

If you look at the Studio and look in the bottom left hand corner, you will see a tab called "Outline". Inside there you will see a list of component names (of components used in the job). If you expand these you will see all of the global variables that can be used for each component. If you take a tJava component and drag the variable from that box into the tJava code, you can use it there in code. It will look something like this.....

((Integer)globalMap.get("tMSSqlOutput_1_NB_LINE_INSERTED"))

[Anonymous]

Hello rhall,

I don't have the Outline tab. I am using Talend Open Studio, it can explain that, I don't know. 

I haven't succeeded to define Global variable in my mapping. Can you help me please. 

Joined : A part of my job with postgresql database

Regards

2017-09-19_114128.jpg

[Anonymous]

A globalMap variable is essentially a value stored in the globalMap HashMap (a data structure). To use this value all you need to know is the name of the component. In the example below, the component is called "tMSSqlOutput_1" the rest of the code below is exactly what you want to use in order to get number of rows inserted.

((Integer)globalMap.get("tMSSqlOutput_1_NB_LINE_INSERTED"))