logging Talend job execution on a Database

[Anonymous]

hi,
could you tell me if it is possible to log Talend job as they process in a database table?
or to a file that can later be imported into a db?
thanks

[Anonymous]

Hello,

Yes, you can use the tStatCatcher and link it to a DBG Output component.
Regards,
Michaël

[Anonymous]

What is the structure of the talend_logs, talend_stats and talend_meters tables to log execution events inside a database ?
How is it possible to use the "Stats & Logs" tabs in the properties panel of each job ?
Is it related to the option in the Talend/preferences ?

[Anonymous]

I need to extract basic information about a job every time its run and output this into a DB.
I need information such as the following:
Username - of the person running the Job
Project Name
Job Name
Start Time
End Time
Row Count
Result ? Success or Failure.
What would I create this Job very simply?

[Anonymous]

Hi dna1718
Username: the job don't know who will run the job, you can use a tMsgbox to ask for the username at the beginning of job.
Project Name: there is a global variable called projectName, you can use this variable.
Job Name: there is a global variable called jobName!
Start Time and End Time: use tChronometerStart and tChronometerStop components to populate the job start time and end time. (see demo in topic: http://www.talendforge.org/forum/viewtopic.php?id=10032)
Row Count: use tFlowMeter and tFlowMeterCapture to populate the number of specific row.
At the end of job, use a tFlowInputFlow to generate one row including these columns username, project name, job name, start time, end time, and row count.
Best regards
Shong

[Anonymous]

Thanks for that. Much appreciated.
Do you know the variable that holds the row count?

[Anonymous]

Thanks for that. Much appreciated.
Do you know the variable that holds the row count?

Please learn tFlowMeter and tFlowMeterCapture components in the user documentation.
Best regards
Shong

[crtatnall]

Several questions about logging:
(1) What is the best way to log events that aren't considered component failures by Talend --- for instance apply functional validations to input data that wouldn't be considered a schema failure per se, or check to make sure the expected number of output records was generated based on the input, etc. ? Should I use tJava or tJavaRec components to manually enforce those edits, and if so how would I get my error messages to tLogCatcher ?
(2) What is best way to have multiple levels of verbosity in my log output ? Is there an input parameter to explicitly tell the job to generate "Debug" level warning messages vs. just regular ones? Is there any way to code the job such that it automatically generates Debug level messages if any errors are encountered, or do I have to code my shell script to re-run the job in Debug mode if an error was encountered the first time?
(3) In general, is it best practices to rely on tWarn and tDie components to generate ALL of my log output, or is there a more elegant/efficient way to code that ? I know that sounds like a strange question given all the documentation on logging, but I thought I heard a Talend consultant once say something that made me think that those should not be over-utilized...
Thanks!
Chris