How to capture each component processed time in Job

[manodwhb]

Hi,

could you please let me know how to capture each component processed time in Job?

Thanks,

Manohar

[vapukov]

You can check 

Stats and Logs (tStatsCatcher)
and
How to automate the use of statistics & logs

[manodwhb]

Since i tried to get the component statistics but not able to get. are you able to get?

[vapukov]

may be we talking about different, but I can not understand - what You are looking for

.-------------------+------+--------+----------+--------+-------------+-------+----------+-------+----------+-----------------------+-----------+------------+--------.
|                                                                              tLogRow_1                                                                              |
|=------------------+------+--------+----------+--------+-------------+-------+----------+-------+----------+-----------------------+-----------+------------+-------=|
|moment             |pid   |root_pid|father_pid|project |job          |context|origin    |message|system_pid|job_repository_id      |job_version|message_type|duration|
|=------------------+------+--------+----------+--------+-------------+-------+----------+-------+----------+-----------------------+-----------+------------+-------=|
|2018-06-18 22:07:58|YjJ08T|YjJ08T  |YjJ08T    |BD_DEMOS|tStatCatcher1|demo   |tUniqRow_1|success|4614      |_CAQbYOdtEdu08_wAFUaozw|0.1        |end         |6       |
'-------------------+------+--------+----------+--------+-------------+-------+----------+-------+----------+-----------------------+-----------+------------+--------'

.-------------------+------+--------+----------+--------+-------------+-------+----------------------+-------+----------+-----------------------+-----------+------------+--------.
|                                                                                    tLogRow_1                                                                                    |
|=------------------+------+--------+----------+--------+-------------+-------+----------------------+-------+----------+-----------------------+-----------+------------+-------=|
|moment             |pid   |root_pid|father_pid|project |job          |context|origin                |message|system_pid|job_repository_id      |job_version|message_type|duration|
|=------------------+------+--------+----------+--------+-------------+-------+----------------------+-------+----------+-----------------------+-----------+------------+-------=|
|2018-06-18 22:07:58|YjJ08T|YjJ08T  |YjJ08T    |BD_DEMOS|tStatCatcher1|demo   |tFileOutputDelimited_1|success|4614      |_CAQbYOdtEdu08_wAFUaozw|0.1        |end         |9       |
'-------------------+------+--------+----------+--------+-------------+-------+----------------------+-------+----------+-----------------------+-----------+------------+--------'

this is just simple text example, but You can store stats in database and make analysis after

[manodwhb]

I am looking for something to capture each component taking time in a job.

[vapukov]

Stats & Logs
When you execute a Job, you can monitor the execution through the tStatCatcher Statistics option or through using a log component. This will enable you to store the collected log data in .csv files or in a database.

You can then set up the path to the log file and/or database once for good in the [Project Settings] dialog box so that the log data get always stored in this location.

Duration: Time for the execution of a Job or a component with the tStatCatcher Statistics check box selected.

this is cover - most of cases, if You have something more important - use tChronometerStart/Stop

[Anonymous]

Hello,

Are you looking for detailed logging of the components? The INFO messages shows the progress of the component? Those detailed logging you were getting with older versions where more of a DEBUG level, but were incorrectly logged with INFO level. So changes have been made to rectify this in newer releases.

Either use Debug level or consider using AMC if you want to track statistics on components.  

Best regards

Sabrina

[Anonymous]

hello, i think this article should help you
-https://www.talend.com/blog/2016/10/05/talend-job-design-patterns-best-practices-part-3/
good day!

[Anonymous]

hello,

I am using tStatCatcher for capturing the log in a flat file.

best regards

[Anonymous]

I used tChronometerStart/Stop to measure the time taken by my job and it works perfectly fine but is it there a way I can store it in a flat file?
Thanks.