topic Re: Best Practice regarding reading CSV file and duplicates in Talend Studio

Best Practice regarding reading CSV file and duplicates

JSey — Sat, 16 Nov 2024 03:35:01 GMT

I was hoping I could get some suggestions for best practice. A system generates multiple CSV for applications, each time with a different name based on time. I understand I can use tFileList to read from multiple CSV. I need to read these CSV's and insert them into Salesforce (new records, not update). How can I ensure I am not creating duplicate records in Salesforce by not reading from the same CSV multiple times?

Re: Best Practice regarding reading CSV file and duplicates

TRF — Tue, 14 Jan 2020 21:51:54 GMT

Rename or move a file as soon as you have finish with it. This the simplest approach.

Re: Best Practice regarding reading CSV file and duplicates

JSey — Tue, 14 Jan 2020 21:55:31 GMT

Thanks for the suggestion. I looked it up, fFileCopy and remove source file should work.

https://community.talend.com/t5/Design-and-Development/Move-File-from-one-folder-to-another-folder/td-p/128103

Re: Best Practice regarding reading CSV file and duplicates

TRF — Tue, 14 Jan 2020 21:59:57 GMT

That's it, the same component gives you the choice.