:: Talend ETL Exception Handling & Customized Logging ::

[_AnonymousUser]

Hi Sabrina,
I new to the Talend ETL tool i have a requirement as below
1. I want to do it a customized logging for all transactions which ever get fails.
2. When i will run job if founds any DB exception then its going to dis connect and not handling Exception/Error
I want to handle it
Please help me out i need your suggestion ASAP, much appreciated.

[_AnonymousUser]

Hi Sabrina,
I am new to the Talend ETL tool i have a requirement as below
1. I want to do it a customized logging for all transactions which ever get fails.
2. When i will run job if founds any DB exception then its going to dis connect and not handling Exception/Error
I want to handle it
Please help me out i need your suggestion ASAP, much appreciated.

[Anonymous]

Hi,

1. I want to do it a customized logging for all transactions which ever get fails.

Are you looking for talend Stats & Logs TalendHelpCentertats & Logs or error component, such as TalendHelpCenter:tLogCatcher which fetches set fields and messages from Java Exception, tDie and/or tWarn and passes them on to the next component?

2. When i will run job if founds any DB exception then its going to dis connect and not handling Exception/Error
I want to handle it

.
Do you want to check DB error exception detail and location to fix it?
Best regards
Sabrina

[_AnonymousUser]

Hi,

1. I want to do it a customized logging for all transactions which ever get fails.

Are you looking for talend Stats & Logs TalendHelpCentertats & Logs or error component, such as TalendHelpCenter:tLogCatcher which fetches set fields and messages from Java Exception, tDie and/or tWarn and passes them on to the next component?

2. When i will run job if founds any DB exception then its going to dis connect and not handling Exception/Error
I want to handle it

.
Do you want to check DB error exception detail and location to fix it?

Best regards
Sabrina

Thanks Sabrina,
What i want is, let say i have 1000 records in one DB(Schema) and i want to insert these data in to other DB(Schema)
in these transactions if 2nd record got any exception its stopping other transaction as well, i want it to handle in such a way that even exception arise it should ignore that record and move to another transaction.
in logs i want to print transaction details for failed cases for identifying and fixing.
Please share if u have any sample of above scenarios. Thanks
Regards,
Rizwan

[Anonymous]

Hi,
Make sure the "die on error" option is unchecked and you can drag a "rejected" row from t<DB>output component to identify the DB exception.
To make it clear, I have designed a demo job: tfileinputdelimited-->tMysqlOut-->reject-->tlogrow
The input file is:
id;name
5;sabrina
6;elise
7;john
3;marry
a;shong
The field of "id" is integer.
Please see my screenshots
Best regards
Sabrina

[Kuntal]

Hi,
I'm new to talend and I'm trying to create an error capturing mechanism on structural validation of a flat file.Need your help to work on this.
Let me use the data example given by Sabrina.
The input file is:
id;name
5;sabrina
6;elise@#
7;john
b;ma!rry
a;shong
Now I need to apply a logic to check the "id" field should not have any alphanumeric value and "name" field shouldn't have any special character.
So for the above data set my good file should have below record
5;sabrina
7;john
And the reject file should have the below records
6;elise@#
b;ma!rry
a;shong
And I would like to have an error file/table where I would like to see something like below 
rec_key|record|error_code|error_msg
6|6;elise@#|ALPHA_CHECK_FAILED|name has a bad value "elise@#"
b|b;ma!rry|NUMERIC_CHECK_FAILED|id has a bad value "b"
b|b;ma!rry|ALPHA_CHECK_FAILED|name has a bad value "b;ma!rry"
a|a;shong|NUMERIC_CHECK_FAILED|id has a bad value "a"

Please let me know how this can be done with talend components. 

[Anonymous]

Hi,
you are on the right track, just play with it a little bit.. (please don't ask to do your homework though).
Have a look at the flow that Sarina has posted, just extend the "rejected" flow to the tFileOutput component..
g.

[Kuntal]

Hi,
 I think I've done fair bit of homework before drafting this post.If you have read my requirement properly you would easily understand that simply extending the reject flow to some file will not serve my purpose because the error message thrown by talend is too superficial and nobody would understand where exactly the error is and what exactly the error is !!
I'm trying to find a way to edit this error message and add an error code which can be easily understood by people involved in analyzing the error table/file.
Moroever if a record has say two fields (as given in the example) and in both the fields if there are some validation issue (malformed data) I would like to capture both ( b;ma!rry >>>>  b|b;ma!rry|NUMERIC_CHECK_FAILED|id has a bad value "b"
b|b;ma!rry|ALPHA_CHECK_FAILED|name has a bad value "b;ma!rry") and not just the first error.

Hope it clears my intention.

[jpf15310]

Why aren't you using Apache Decanter: it will pretty give you not only the error, but the payload as well.
JP

[JohnnyBravo]

Hi team,
I am new to talend and there is task that I need help, I want to centralize the error handling meaning if any Job throws an error I want one specific error job to kick off and report the error. I dont want to have following mapping  t-logcatcher --->t_Map--->t_Output to be called in every mapping. Is there a way to achieve this in talend