topic Re: Reject Handling - best practices in Talend Studio

Reject Handling - best practices

Anonymous — Sat, 16 Nov 2024 13:33:01 GMT

I wonder if someone could tell me if there is a better way to accomplish what I'm trying to do. I'm having trouble specifying a specific reject reason for a data set when a lookup fails in the tMap component (if a tMap contains several lookups and all are inner joins, then there is apparently no way to know which lookup failed and caused the record to be rejected). My solution is to use a series of tMaps and each tMap only contains one lookup - which makes it easy to add a column to the reject output and specify a reject reason : i.e. : "Invalid Customer Number!" when the Customer Number tMap fails.
My solution is to use one tMap component for each lookup. This makes it possible to know why the lookup failed and append a "reject reason" field to the output. I have included a screenshot of the relevant job flow which shows how the data is flowing through a series of tMaps, with a tFileOutput for the rejects. This seems to be a cumbersome way of doing it, and now I have to find a way to merge all of those files back together (tUnite does not work in this case for some reason).
Is it possible to identify which lookup a record failed on in a tMap component which contains several lookups?
Here's an example:
If I have input with three fields:
Customer Number, Customer Type, Region
and I want to perform lookups on all 3 fields so that I can convert each into an ID field (as is normal with referential integrity), like this:
CUSTOMER_ID, TYPE_ID, REGION_ID
Is it possible to do all of this in one tMap AND also to know for example if a record is rejected whether it was rejected due to Invalid Customer Number, Invalid Type, or Invalid Region?
It seems to me like if we do it all in one tMap then when a record is rejected we won't be able to know what was the specific reason that it failed.
Thanks,
Joe

Re: Reject Handling - best practices

alevy — Thu, 04 Mar 2010 07:21:49 GMT

tUnite does not work in this case for some reason

Talend does not allow flows to be split and rejoined e.g. two flows originating from one tMap component to be re-united at a tUnite or tFileOutput). However, you could possibly have each output component reference the same file; although that might only work with text files.

Is it possible to do all of this in one tMap AND also to know for example if a record is rejected whether it was rejected due to Invalid Customer Number, Invalid Type, or Invalid Region?

With tMap inner joins, Talend does not attempt any joins after the one that fails. You can thus have all your lookups in one tMap with one table of rejects but with the reason determined by which of the lookup fields have a value and which are null e.g. in Java syntax, assuming the lookup tables are in the order you've given:
CustomerTable.CUSTOMER_ID==null
?"Invalid Customer Number!"
:CustomerTypeTable.TYPE_ID==null
?"Invalid Customer Type!"
:"Invalid Region!"

Re: Reject Handling - best practices

Anonymous — Wed, 17 Mar 2010 13:03:29 GMT

Thank you, alevy.

Re: Reject Handling - best practices

_AnonymousUser — Mon, 05 Aug 2013 08:12:17 GMT

Hi

I have created a job with following flow :
tfilelist -->tfileinputdelimited--------->tmap-->tmysqloutptutbulkexec
|reject
|--> tfileoutputdelimited

the data rejected from tfileinputdelimited will be got in tfileoutputdelimited with reason of rejection
The problem is the output file does not contain entire row details,if the row is rejected because of first column data then the output file contains only that field value in the row with reason for rejection . But i want the entire row values data for reloading purpose how to achieve that .

Re: Reject Handling - best practices

Anonymous — Mon, 05 Aug 2013 08:17:59 GMT

Hi banu,
What's your expected result and actual result? Why did you use rejects row from tfileinputdelimited instead of tMap? Could you please elaborate your case with an example?
Best regards
Sabrina

Re: Reject Handling - best practices

_AnonymousUser — Mon, 05 Aug 2013 08:30:57 GMT

Hi ,

The input file contains some junk value that gets rejected in tfileinputdelimited component only
the rejected file contains below value:
;;;For input string: "slu1a^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@0009102451^@0009109438^@1357050300^@1357051200^@0000006988^@0009102451" - Line: 0

What actually i want is the rejected file should contain entire row values like as follows:
slu1a^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@0009102451^@0009109438^@1357050300^@1357051200^@0000006988^@0009102451;565;767;76 then rejected reason .

Re: Reject Handling - best practices

Anonymous — Mon, 05 Aug 2013 09:29:59 GMT

Hi,
What's the rejected reason? Usually, the rejects row from input file can be shown the string (see my screenshot for data type issue).
Best regards
Sarina

Re: Reject Handling - best practices

_AnonymousUser — Mon, 05 Aug 2013 09:34:10 GMT

hi sabrina,

what i want is ,the data should be written into the file with all the column values not only till the column being rejected ....

Re: Reject Handling - best practices

Anonymous — Mon, 05 Aug 2013 10:34:26 GMT

Hi,
So far, the rejects information just show the error string and the location.
For example,
Id;Name;Age;Email
1;shong;11;shong@talend.com
a;elise;22;elise@talend.com
We set Id as "int/integer" and the string "a" will be rejected.
The error message will be : For input string "a"-Line 2 instead of For input string "a;elise;22;elise@talend.com" -Line2.
Best regards
Sabrina

Re: Reject Handling - best practices

_AnonymousUser — Mon, 05 Aug 2013 11:12:23 GMT

thx for reply

Re: Reject Handling - best practices

Anonymous — Tue, 16 Dec 2014 14:50:03 GMT

Hi Sabrina,
I'm trying to save the reject data into a table on oracle. But no matter what output that i use, always come back the line with error as primary key. See my example:
I have a table with 3 columns: id, name, email

1	NAME 1	name1@email.com
- Line: 0
2	NAME 2	name1@email.com
- Line: 1
3	NAME 3	name1@email.com
- Line: 2
4	NAME 4	name1@email.com
- Line: 3

So, when i tried to save the data, i have an error with ID, because is not a number. What can i do to take it off the line with "Line errors"?

Re: Reject Handling - best practices

Anonymous — Wed, 17 Dec 2014 07:19:26 GMT

Hi viniciuscoimbra,
Could you please give us more info about your current job?
Best regards
Sabrina