topic Re: [resolved] tXMLMap - catch errors/exceptions in Talend Studio

[resolved] tXMLMap - catch errors/exceptions

Anonymous — Fri, 27 Nov 2015 02:50:50 GMT

Hi guys,
I have a small Talend job that reads CSVs and makes an XML from its rows, then moves the CSVs in an archive folder afterwards.
Im having a problem trying to catch CSVs with errors, the job stops and doesnt continue on the other CSVs afterwards.
How can I catch the errors in the XML mapping and be able to move that certain CSV, then continue with the other CSVs in processing them?

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Fri, 27 Nov 2015 03:21:42 GMT

Hi
You need to design the job as below:
main job:
tFileList--iterate--trunjob
trunJob: call the child job, pass the current file path to the child job, uncheck the 'die on error' option so that the main job can continue to process the next file even though the child job has an error.
child job:
tFileInputDelimited--main--tXMLMap--tLogRow
Fefer to this KB article to know how to pass a value from main job to child job.
Let me know if you have any problems.

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Fri, 27 Nov 2015 03:24:26 GMT

Does this mean I will have 2 Talend Jobs now?
Is there another workaround that only involved 1 job? Because this will be built as a standalone job that will be schedule using a script.

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Fri, 27 Nov 2015 03:29:34 GMT

Hi
Strictly speaking, they are not entirely separate jobs, they are not entirely separate jobs, the child job is called by the main job, you just need to export the job script of main job as a standalone job and schedule it.

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Fri, 27 Nov 2015 04:23:11 GMT

Hello,
Unfortunately my manager/in-charge does not want the job to change that much anymore, he just wants to have a way to continue processing other CSV rows and files after encountering an error in a certain CSV. Would this be possible without altering the job that much?
Can I catch/bypass a row/file error and continue with the other rows and files in the folder?

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Fri, 27 Nov 2015 09:10:16 GMT

Hi
If the error occurs on tFileInputDelimited component, unchecking the 'die on error' option on tFileInputDelimited will allow the job to continue to read other rows and files, but the error messages can't be captured by tLogCatcher component with this way.

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Fri, 27 Nov 2015 09:25:28 GMT

The error is in the tXMLMap, when it attempts to substring invalid/null fields. How can I catch those?

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Fri, 27 Nov 2015 10:20:32 GMT

There is a way you can trap the errors in the tXMLMap and continue...well, actually you will need to add a tMap before the tXMLMap and put the logic from the tXMLMap into the tMap. This is because for some reason the functionality I demonstrate in this link is not available in the tXMLMap.
Why is that Shong?

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Fri, 27 Nov 2015 10:55:08 GMT

Hi rhall_2.0
I agree with you that tMap can be used to trace the error and continue to process the next rows if the error is not an I/O error occurred on input/output component.

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Fri, 27 Nov 2015 11:07:16 GMT

Why is that functionality not available in the tXMLMap?

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Sat, 28 Nov 2015 04:26:15 GMT

A new feature has been reported in bugtracker, see
https://jira.talendforge.org/browse/TDI-18968

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Mon, 30 Nov 2015 09:37:16 GMT

I see, so my job layout should now be . . .
Before:
tFileList --(iterate)-- iFileInputDelimited --(row)-- tXMLMap
After:
tFileList --(iterate)-- iFileInputDelimited --(row)-- tMap --(row)-- tXMLMap
Where tMap now has the formatting codes and tXMLMap is merely assigning the fields to their respective XML elements, is that correct?

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Mon, 30 Nov 2015 09:56:06 GMT

Yes, that is the way that I would attempt this to make use of the tMap error logging functionality.

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Tue, 01 Dec 2015 10:22:28 GMT

Okay thanks for the suggestion, will try this out and update this topic accordingly.

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Thu, 03 Dec 2015 08:07:04 GMT

Hello,
I am now able to continue after encountering a row error. Thanks rhall_2.0 and shong.
Unfortunately if consecutive errors are shown as 1 when I try to print something in the ErrorReject row.
Also, it ignores the first okay row after encountering an error row/group of error rows.
For example, this is my csv file:
- headers
- ok row
- ok row
- ok row
- error row
- error row
- ok row
- ok row
- ok row
- error row
- error row
- error row
- ok row
- ok row
It prints the following:
- ok
- ok
- ok
- error (should be 2 times)
- ok
- ok (should be 3 times together with previous print)
- error (should be 3 times)
- ok (should be 2 times)
How can I make it print "error" for each instance of the error row, and not as a group of consecutive error rows?

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Thu, 03 Dec 2015 09:43:06 GMT

This is very odd. Can you take a screenshot of your job, your tMap configuration and the following tXMLMap config?

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Thu, 03 Dec 2015 09:55:56 GMT

I find it very weird also, I was expecting it to correctly print according to the ok/error row.
Anyways here is my job layout:

My tMap component has the parsing and formatting conditions, with the die on error unticked:

And my tXMLMap is a straight auto-map into the XML structure:

Is there some hidden or inconspicuous setting that I might have missed or is this some Talend bug?

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Thu, 03 Dec 2015 10:08:26 GMT

Where is the code written in the tJavaFlex component logging the errors?
In your tXMLMap output table settings (click the spanner symbol), do you have "All in one" selected? This allows one XML file to be written containing several loops as opposed to a separate XML for every row.
Maybe try connecting you tMap outputs to tLogRow components (instead of the tJavaFlex and tXMLMap) and and see what comes out from there. You should be seeing something for every row.

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Thu, 03 Dec 2015 10:16:26 GMT

Where is the code written in the tJavaFlex component logging the errors?
It is in the tJavaFlex_7, the one connected with ErrorReject row from the tMap
In your tXMLMap output table settings (click the spanner symbol), do you have "All in one" selected? This allows one XML file to be written containing several loops as opposed to a separate XML for every row.
I have set "All in one" into false because I have to generate 1 XML per row read from the CSV.
Maybe try connecting you tMap outputs to tLogRow components (instead of the tJavaFlex and tXMLMap) and and see what comes out from there. You should be seeing something for every row.
Only 1 instance of tLogRow output gets printed instead of the 2 or 3 consecutive for the error rows. Same goes for the ok rows.

Re: [resolved] tXMLMap - catch errors/exceptions

Anonymous — Thu, 03 Dec 2015 10:25:12 GMT

ADDITIONAL:
My job also shows that it processed 8 rows, with 6 ok and 2 error when instead it should be 13 rows with 8 ok and 5 errors. No limit has been set.