topic How to read the data from the failure in Talend Studio

How to read the data from the failure

Anonymous — Wed, 25 Oct 2017 05:04:00 GMT

Hi,

I have a requirement. In my source we have 1000(example) records and performing logic and loading into database.

The job failed after loading 450 records. Now I want to start loading/ reading the source file from 451 to 1000 and It should not read 1-450(which already loaded). Please suggest how can we achieve this using talend.

My Job:

tfileinputdeleimeted->tmap->toracledb.

Thanks in advance.

Re: How to read the data from the failure

TRF — Wed, 25 Oct 2017 07:52:38 GMT

Hi,

There is a lot of way for this.

If you just want to ignore the n first records from your input file (that's your request), you may have a context variable set to 1 by default ignore the header if any (set to 0 in case of no header), and set the number of records to ignore (450 in your example) + 1 if you have a header.

Then, use this variable to populate the tFileInputDelimited "Header" field.

That's all.

Re: How to read the data from the failure

Anonymous — Wed, 25 Oct 2017 08:26:12 GMT

Hi,

There are two ways to achieve this.

1) If you have a unique key in your delimited file, then you can apply a join (inner join) between the file and the database table in which you are inserting the records. That can be done through tMap. In this way it will escape all the records that are already inserted into the table.

2) Its not the ideal way, but you can set the header value to 451 (If you have headers) OR 450 (If you don't have headers). In this way it will escape the first 450 records.

Hope it will help you in resolving your query !

Best Regards,

Abhishek

Re: How to read the data from the failure

Anonymous — Wed, 25 Oct 2017 09:12:35 GMT

Hi there,

From the information you've provided, and assuming this is a one-off exercise, you just need to temporarily change the Header property in your tFileInputDelimited to 450, as suggested by @TRF ad @abhishek.

If there are actual header lines in the source file, then you'll also need to add the number of header lines, so e.g. if you have a two line header, the Header value should be 452.

Regards,

Chris

Re: How to read the data from the failure

Anonymous — Wed, 25 Oct 2017 09:20:20 GMT

Thanks all for your reply..giving the value at header resolved the issue.

Re: How to read the data from the failure

Anonymous — Wed, 25 Oct 2017 09:39:38 GMT

I notice that you have 3 components with a tMap being in the middle. Therefore it is likely that your tMap is where the error occurs. Here is a tutorial on how you can trap the error row in a tMap (https://www.rilhia.com/quicktips/quick-tip-how-debug-tmap-errors).

Re: How to read the data from the failure

TRF — Wed, 25 Oct 2017 18:40:23 GMT

Did you found an answer to your initial question?
If so, mark your case as solved by choosing the appropriate answer.