topic Re: Only 1st column in tMap CSV file valued for constraint in Talend Studio

Only 1st column in tMap CSV file valued for constraint

Anonymous — Sat, 16 Nov 2024 02:44:12 GMT

Hello,

Thanks for your help in advance.

I'm having an issue where only the 1st column of my CSV file is being considered for constraints.

I've setup tMap (attached) to pick up erroneous rows and reject them into a Reject.CSV file.

When the job is run and a constraint column other than column 1 is found, it gives me an Oracle error.

Not sure why the tMap isn't picking up other columns. Please see attached.

Thanks

Re: Only 1st column in tMap CSV file valued for constraint

manodwhb — Fri, 10 Apr 2020 09:38:08 GMT

What error are you getting?

Re: Only 1st column in tMap CSV file valued for constraint

Anonymous — Fri, 10 Apr 2020 12:48:38 GMT

@manodwhb the error is an Oracle error that I cannot enter a NULL value is a non-null column. But the columns I have specified in "LOADED" connection in tMap should all be checked for NULL values, and they should be sent to REJECTED connection if they are NULL, or am I missing something here? But this is only happening for the first column which works fine, the rest of the columns are not checked if they are NULL and therefore I'm getting errors. Please take a look at the first posting for the screenshots.

Re: Only 1st column in tMap CSV file valued for constraint

manodwhb — Fri, 10 Apr 2020 12:51:41 GMT

How are you checking the null validation? Are you writing some expression enable or disable filter expression in the output of tmap ?

Re: Only 1st column in tMap CSV file valued for constraint

Anonymous — Fri, 10 Apr 2020 12:57:37 GMT

@manodwhb No, I just have the "Loaded" tMap connection transferring the columns with the same constraints as I have in the Oracle table. The "ErrorReject" connection (I disabled Die on Error) has the same setup which is where it should create the CSV file with rejects. I don't have any expressions to check, do I need to? I thought that the "Die on Error" and "ErrorReject" options would accommodate this fact.

Thanks again for your help.

Re: Only 1st column in tMap CSV file valued for constraint

manodwhb — Fri, 10 Apr 2020 13:10:40 GMT

How does it validate the constrian ? Take loaded one output from the tmap and connect tdboutput and take the reject from tdboutput.

Re: Only 1st column in tMap CSV file valued for constraint

Anonymous — Fri, 10 Apr 2020 13:22:26 GMT

@manodwhb Sorry, should have added screenshots embedded in the message.

Here is my current setup. How do you propose I should adjust it?

Thanks

Re: Only 1st column in tMap CSV file valued for constraint

manodwhb — Fri, 10 Apr 2020 13:25:35 GMT

In tmap only have loaded one and connect to tdboutput. From the tdboutput take reject link and connect to tfileoutputdelimited.

Re: Only 1st column in tMap CSV file valued for constraint

Anonymous — Fri, 10 Apr 2020 13:34:59 GMT

@manodwhb that would make much more sense. I can't seem to find the Reject link at tdboutput though.

Re: Only 1st column in tMap CSV file valued for constraint

manodwhb — Fri, 10 Apr 2020 13:35:39 GMT

Are you using enterprise Talend?

Re: Only 1st column in tMap CSV file valued for constraint

Anonymous — Fri, 10 Apr 2020 13:36:25 GMT

@manodwhb Talend Open Studio for Data Integration

Re: Only 1st column in tMap CSV file valued for constraint

manodwhb — Fri, 10 Apr 2020 13:40:17 GMT

OK. What is your constraint ,on how many columns it is there?

Re: Only 1st column in tMap CSV file valued for constraint

manodwhb — Fri, 10 Apr 2020 13:44:11 GMT

check the below link will help you how enable filter for the output and get reject data from tmap.
https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781782167266/3/ch03lvl1sec22/rejecting-rows-using-tmap

Re: Only 1st column in tMap CSV file valued for constraint

Anonymous — Fri, 10 Apr 2020 13:58:37 GMT

@manodwhb thanks for the link. This will filter based on the source CSV data and tMap would filter based on its own rules. However, I was hoping to have all rejects which were denied entry into the Oracle table including incorrect data type, incorrect data format, etc...

Re: Only 1st column in tMap CSV file valued for constraint

manodwhb — Fri, 10 Apr 2020 14:52:16 GMT

Since you're using open studio you need to do manual check only.

Re: Only 1st column in tMap CSV file valued for constraint

Anonymous — Fri, 10 Apr 2020 14:58:39 GMT

@manodwhb Thanks for your help. It seems Oracle tDB output is the culprit as other outputs do have the reject option. Nevertheless, thanks for your time!