I have a folder of excel files Talend package reads the folder and for each excel file in the folder it unpivots the content of the excel file and inserts in a MSSQL database. Now the same row/record could exist in multiple files and thats why we keep getting duplicates in the database. if someone could help me understand, if there is a way to check if the record exists in MSSQL DB, before inserting then i can skip duplicates and just get unique records. Please see the attached screenshot. Thanks
Can you add a unique key to your database table? If you then set the MSSQL component to be "Insert if not exist", this would stop duplicates being inserted.
Thank You. Very Helpful. What should i do if i want to update the existing record instead of skipping it? Is there an option in the tMSSQL component for this as well?
OK - set the action on data to be "Update or insert". This updates existing entries or create it if non existing.
Hi,
For the Update or insert otpion in mysql output it requires a primary key. So how to get the primary keys from db and assign it to the rows in excel sheet. Kindly help.
Hi,
So far, there is no "insert if not exist" option in tverticaoutput.
As alternative, you can set any primary key you wanted in schema setting by manual(checkout the "Key" option) .
and take the "Update or insert" data action in tverticaoutput.
Best regards
Sabrina
Hi,
I want to convert excel sheet to mysql table.MySQl table has id which is auto increment.
I have list of user in excel sheet which might contain duplicate.I have Insert ignore option which give me correct record but when add new record in existing excel file . auto increment value is almost double. for example
I have list of user 20 distinct values first time in excel file but i have added one more and try to run same excel file . the auto increment value for new record will be 41 but i want 21. is it posible?
please let me know..its very urgent.
Thank you in advance.
