raised new topic

[Anonymous]

I have used tFilelist to iterate over csv files and process them one by one to database table.

Here I have observed if there is error in any one file processing then the remaining files does not get processed and job gets stuck.

I want the tFilelist iterator to continue with next file after there is error in one csv file.

[Anonymous]

Hi,

    Could you please do a schema compliance check to remove all the bad records before loading to DB. In this way, you can avoid the failure due to data errors.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[Anonymous]

Appriciate your answer but what if there is constraint violation while insert?? data is fine and validated.

[Anonymous]

Hi,

    Since you are truncating the table, you will not have any issues due to previous data. Now in the current data set, if you are having duplicate records, you can take Unique records from them before doing insert itself by using tUniqrow component.

    You will have to select the keys from the columns in this component and it will segregate the uniques and duplicates.

    Now, even if you are having existing data in your table, you can select insert or update option in "Action on Data" of tDBOutput component and it also will resolve your issue. 

Hope I have answered your queries for your topic. Before going, please mark the topic as resolved so that other Talend members also will be benefited from the topic.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[Anonymous]

Hi Nikhil,
I don't have to update record with new one. so this solution is not applicable for me anyways Thank You.

[Anonymous]

Hi Roshan,

     This requirement was not clear in your previous posts. If you do not want to update any existing record, its quite simple.

      Why don't you do a tMap join where lookup is your target table? If you are doing Inner join and have selected inner join reject as True in output section, then only new records will be selected to the output flow.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂