topic Re: :: Talend ETL Exception Handling & Customized Logging :: in Talend Studio

:: Talend ETL Exception Handling & Customized Logging ::

_AnonymousUser — Wed, 21 May 2014 06:21:18 GMT

Hi Sabrina,
I new to the Talend ETL tool i have a requirement as below
1. I want to do it a customized logging for all transactions which ever get fails.
2. When i will run job if founds any DB exception then its going to dis connect and not handling Exception/Error
I want to handle it
Please help me out i need your suggestion ASAP, much appreciated.

Re: :: Talend ETL Exception Handling & Customized Logging ::

_AnonymousUser — Wed, 21 May 2014 06:22:12 GMT

Hi Sabrina,
I am new to the Talend ETL tool i have a requirement as below
1. I want to do it a customized logging for all transactions which ever get fails.
2. When i will run job if founds any DB exception then its going to dis connect and not handling Exception/Error
I want to handle it
Please help me out i need your suggestion ASAP, much appreciated.

Re: :: Talend ETL Exception Handling & Customized Logging ::

Anonymous — Wed, 21 May 2014 07:22:47 GMT

Hi,

1. I want to do it a customized logging for all transactions which ever get fails.

Are you looking for talend Stats & Logs TalendHelpCentertats & Logs or error component, such as TalendHelpCenter:tLogCatcher which fetches set fields and messages from Java Exception, tDie and/or tWarn and passes them on to the next component?

2. When i will run job if founds any DB exception then its going to dis connect and not handling Exception/Error
I want to handle it

.
Do you want to check DB error exception detail and location to fix it?
Best regards
Sabrina

Re: :: Talend ETL Exception Handling & Customized Logging ::

_AnonymousUser — Wed, 21 May 2014 07:41:45 GMT

Hi,

1. I want to do it a customized logging for all transactions which ever get fails.

Are you looking for talend Stats & Logs TalendHelpCentertats & Logs or error component, such as TalendHelpCenter:tLogCatcher which fetches set fields and messages from Java Exception, tDie and/or tWarn and passes them on to the next component?

2. When i will run job if founds any DB exception then its going to dis connect and not handling Exception/Error
I want to handle it

.
Do you want to check DB error exception detail and location to fix it?

Best regards
Sabrina

Thanks Sabrina,
What i want is, let say i have 1000 records in one DB(Schema) and i want to insert these data in to other DB(Schema)
in these transactions if 2nd record got any exception its stopping other transaction as well, i want it to handle in such a way that even exception arise it should ignore that record and move to another transaction.
in logs i want to print transaction details for failed cases for identifying and fixing.
Please share if u have any sample of above scenarios. Thanks
Regards,
Rizwan

Re: :: Talend ETL Exception Handling & Customized Logging ::

Anonymous — Wed, 21 May 2014 08:06:50 GMT

Hi,
Make sure the "die on error" option is unchecked and you can drag a "rejected" row from t<DB>output component to identify the DB exception.
To make it clear, I have designed a demo job: tfileinputdelimited-->tMysqlOut-->reject-->tlogrow
The input file is:
id;name
5;sabrina
6;elise
7;john
3;marry
a;shong
The field of "id" is integer.
Please see my screenshots
Best regards
Sabrina

Re: :: Talend ETL Exception Handling & Customized Logging ::

Kuntal — Thu, 17 Dec 2015 12:05:43 GMT

Hi,
I'm new to talend and I'm trying to create an error capturing mechanism on structural validation of a flat file.Need your help to work on this.
Let me use the data example given by Sabrina.
The input file is:
id;name
5;sabrina
6;elise@#
7;john
b;ma!rry
a;shong
Now I need to apply a logic to check the "id" field should not have any alphanumeric value and "name" field shouldn't have any special character.
So for the above data set my good file should have below record
5;sabrina
7;john
And the reject file should have the below records
6;elise@#
b;ma!rry
a;shong
And I would like to have an error file/table where I would like to see something like below
rec_key|record|error_code|error_msg
6|6;elise@#|ALPHA_CHECK_FAILED|name has a bad value "elise@#"
b|b;ma!rry|NUMERIC_CHECK_FAILED|id has a bad value "b"
b|b;ma!rry|ALPHA_CHECK_FAILED|name has a bad value "b;ma!rry"
a|a;shong|NUMERIC_CHECK_FAILED|id has a bad value "a"

Please let me know how this can be done with talend components.

Re: :: Talend ETL Exception Handling & Customized Logging ::

Anonymous — Thu, 17 Dec 2015 20:33:02 GMT

Hi,
you are on the right track, just play with it a little bit.. (please don't ask to do your homework though).
Have a look at the flow that Sarina has posted, just extend the "rejected" flow to the tFileOutput component..
g.

Re: :: Talend ETL Exception Handling & Customized Logging ::

Kuntal — Fri, 18 Dec 2015 07:17:49 GMT

Hi,
I think I've done fair bit of homework before drafting this post.If you have read my requirement properly you would easily understand that simply extending the reject flow to some file will not serve my purpose because the error message thrown by talend is too superficial and nobody would understand where exactly the error is and what exactly the error is !!
I'm trying to find a way to edit this error message and add an error code which can be easily understood by people involved in analyzing the error table/file.
Moroever if a record has say two fields (as given in the example) and in both the fields if there are some validation issue (malformed data) I would like to capture both ( b;ma!rry >>>> b|b;ma!rry|NUMERIC_CHECK_FAILED|id has a bad value "b"
b|b;ma!rry|ALPHA_CHECK_FAILED|name has a bad value "b;ma!rry") and not just the first error.

Hope it clears my intention.

Re: :: Talend ETL Exception Handling & Customized Logging ::

jpf15310 — Sun, 24 Jan 2016 22:31:45 GMT

Why aren't you using Apache Decanter: it will pretty give you not only the error, but the payload as well.
JP

Re: :: Talend ETL Exception Handling & Customized Logging ::

JohnnyBravo — Fri, 05 Feb 2016 07:49:57 GMT

Hi team,
I am new to talend and there is task that I need help, I want to centralize the error handling meaning if any Job throws an error I want one specific error job to kick off and report the error. I dont want to have following mapping t-logcatcher --->t_Map--->t_Output to be called in every mapping. Is there a way to achieve this in talend

Re: :: Talend ETL Exception Handling & Customized Logging ::

Anonymous — Fri, 05 Feb 2016 15:15:39 GMT

emaduddinsyed
tLogCatcher serves as a general try/catch handler, but nobody forces you to log the exception into the tLogRow (or other output), you're free to start your own job
g.

Re: :: Talend ETL Exception Handling & Customized Logging ::

Anonymous — Wed, 05 Apr 2017 07:38:15 GMT

Hi,
I think I've done fair bit of homework before drafting this post.If you have read my requirement properly you would easily understand that simply extending the reject flow to some file will not serve my purpose because the error message thrown by talend is too superficial and nobody would understand where exactly the error is and what exactly the error is !!
I'm trying to find a way to edit this error message and add an error code which can be easily understood by people involved in analyzing the error table/file.
Moroever if a record has say two fields (as given in the example) and in both the fields if there are some validation issue (malformed data) I would like to capture both (b;ma!rry >>>> b|b;ma!rry|NUMERIC_CHECK_FAILED|id has a bad value "b"
b|b;ma!rry|ALPHA_CHECK_FAILED|name has a bad value "b;ma!rry") and not just the first error.

Hope it clears my intention.

Have pretty much the same requirement.
Any help is much appreciated.