topic Re: logging Talend job execution on a Database in Installing and Upgrading

logging Talend job execution on a Database

Anonymous — Sat, 16 Nov 2024 14:35:47 GMT

hi,
could you tell me if it is possible to log Talend job as they process in a database table?
or to a file that can later be imported into a db?
thanks

Re: logging Talend job execution on a Database

Anonymous — Tue, 19 Jun 2007 12:48:48 GMT

Hello,

Yes, you can use the tStatCatcher and link it to a DBG Output component.
Regards,
Michaël

Re: logging Talend job execution on a Database

Anonymous — Tue, 25 Sep 2007 15:13:53 GMT

What is the structure of the talend_logs, talend_stats and talend_meters tables to log execution events inside a database ?
How is it possible to use the "Stats & Logs" tabs in the properties panel of each job ?
Is it related to the option in the Talend/preferences ?

Re: logging Talend job execution on a Database

Anonymous — Wed, 01 Feb 2012 11:32:57 GMT

I need to extract basic information about a job every time its run and output this into a DB.
I need information such as the following:
Username - of the person running the Job
Project Name
Job Name
Start Time
End Time
Row Count
Result ? Success or Failure.
What would I create this Job very simply?

Re: logging Talend job execution on a Database

Anonymous — Fri, 03 Feb 2012 01:41:47 GMT

Hi dna1718
Username: the job don't know who will run the job, you can use a tMsgbox to ask for the username at the beginning of job.
Project Name: there is a global variable called projectName, you can use this variable.
Job Name: there is a global variable called jobName!
Start Time and End Time: use tChronometerStart and tChronometerStop components to populate the job start time and end time. (see demo in topic: http://www.talendforge.org/forum/viewtopic.php?id=10032)
Row Count: use tFlowMeter and tFlowMeterCapture to populate the number of specific row.
At the end of job, use a tFlowInputFlow to generate one row including these columns username, project name, job name, start time, end time, and row count.
Best regards
Shong

Re: logging Talend job execution on a Database

Anonymous — Fri, 03 Feb 2012 09:02:39 GMT

Thanks for that. Much appreciated.
Do you know the variable that holds the row count?

Re: logging Talend job execution on a Database

Anonymous — Fri, 03 Feb 2012 09:29:03 GMT

Thanks for that. Much appreciated.
Do you know the variable that holds the row count?

Please learn tFlowMeter and tFlowMeterCapture components in the user documentation.
Best regards
Shong

Re: logging Talend job execution on a Database

crtatnall — Fri, 28 Mar 2014 22:07:16 GMT

Several questions about logging:
(1) What is the best way to log events that aren't considered component failures by Talend --- for instance apply functional validations to input data that wouldn't be considered a schema failure per se, or check to make sure the expected number of output records was generated based on the input, etc. ? Should I use tJava or tJavaRec components to manually enforce those edits, and if so how would I get my error messages to tLogCatcher ?
(2) What is best way to have multiple levels of verbosity in my log output ? Is there an input parameter to explicitly tell the job to generate "Debug" level warning messages vs. just regular ones? Is there any way to code the job such that it automatically generates Debug level messages if any errors are encountered, or do I have to code my shell script to re-run the job in Debug mode if an error was encountered the first time?
(3) In general, is it best practices to rely on tWarn and tDie components to generate ALL of my log output, or is there a more elegant/efficient way to code that ? I know that sounds like a strange question given all the documentation on logging, but I thought I heard a Talend consultant once say something that made me think that those should not be over-utilized...
Thanks!
Chris