File processing and error handling and continue if file has a bad rows

[Bhaj]

Hi all,

I'm very new to Talend data integration. I'm testing csv file processing to oracle db.

My question is how to catch errors and log into the table, and to continue the file processing even if the file has any data e.g which have a column name larger than defined in table.

Below is my job:

[Anonymous]

Hi

If you want the job continue to process the file, uncheck the 'Die on error' box on the basic settings panel of tOracleComponent, and unchecked the 'Use batch size' box on the advanced settings panel of tOracleComponent, catch the reject flow after tOracleOutput.

....tOracleOutput--reject--tLogRow

Regards

Shong

[Bhaj]

Hi shong,

Thank you very much for your support. It's working as you said, but I have another Issue.

How to move these bad records to another directory in the same filename, and skip row count for these bad records.

Regards,

Bujar

[Anonymous]

Can you show an example of reject rows? Need to find the information of bad records from the reject rows. ​

About the row count, there are predefined global variables you can use directly, such as:

((Integer)globalMap.get("tDBOutput_1_NB_LINE_INSERTED"))

[Bhaj]

Rejected records e.g length of data in csv file is larger than column in destination table, so I want the same files to process data and the bad records to keep in the same file but to move with bad content (remaining bad records in that file).

e.g file1 (id,name) (100,testtesrerer) - table test (id number(2), name varchar2(5))

file1 (id,name) (200,test) - table test (id number(2), name varchar2(5))

in this case I want file1 to process first row, and to keep second row in that file and move file1 (without first row, only bad records) in different directory with ".bad" extension.

[Anonymous]

Link the reject rows to tMap and filter the columns you want, output the result to a file in another folder, eg:

...tOracleOutput--reject--tMap---tFileOutputDelimited

on tFileOutputDelimited, set the file path like:

"d://folder_path/"+((String)globalMap.get("tFileList_1_CURRENT_FILE"))+".bad"