Creating detailed log file

[Anonymous]

Hi All,

I want to create a detailed log/stats file with details such as

Job Name, Source file name, Source file count, Target file name, Target insert count, Time taken for job, start date, end date, Job status, failed status etc and store it either in file or table.

Can anyone help me with the same pls.

Thanks!

[Anonymous]

Hi,

    You can use the logging and error components of Talend.

https://help.talend.com/reader/KxVIhxtXBBFymmkkWJ~O4Q/exEndmX5MGHMto5BeyIMhg

    Please also go through below link.

https://help.talend.com/reader/SAZUuM203k6SNl8zrDfj5Q/GRS3dWpaYoK_QdY~Mvck5Q

    I believe you would also be interested in AMC.

https://talendpnp.github.io/architecture-center/reference-architecture/summer-2018-7.0/talend-compon...

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

[Anonymous]

Appreciate your response @nthampi .

All these components give some columns required.

Is there a way we can capture all at once. Instead of adding so many components. 

The reason being, i want to make it reusable across multiple jobs or all jobs.

[Anonymous]

Hi,

    There is an interesting blog related to this subject.

https://www.talend.com/blog/2018/10/09/how-to-implement-a-job-metadata-framework-using-talend/

    Why don't you add it as a Joblet so that it can be reused easily with other jobs?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

[Anonymous]

Thanks, @nthampi.

I was referring to the same link. However, I'm not sure how to access tables such as talend_file_tracker,talend_jobs, talend_hwm_tracker, etc.

Or if I have to populate these tables, not sure of the components.

Appreciate your help.

[Anonymous]

Hi,

    You will have to setup the necessary tables based on the data extracted from Talend error and log components. The flowchart is depicting the method in which you will have to pass the data from the log components.

     Unfortunately I do not have the code handy with me. I would suggest you to build the code after verifying the elements I have already mentioned in the previous posts. If you are facing any specific error, we can guide you during that point of time.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

[Anonymous]

Thanks @nthampi.

I did try using multiple components and putting it in one table.

However each component offers different feature and can't connect all component and even if i do i can't make it generic.

And i'm still not able to generate those tables, since i'm not sure how to generate some of the column values.

Could you please guide me.

Many thanks!