topic Re: How to ignore 'duplicate key' errors in Talend Studio

How to ignore 'duplicate key' errors

Anonymous — Sat, 16 Nov 2024 13:46:22 GMT

Hello all,
I'm sorry to bother you with stupid question, but I'm new to TOS and I'm not able to find anything similar in both UG and this forum.
I want to pump data from a MySQL schema to an other MySQL schema, ignoring duplicate key errors on destination table.
I gave a try to all available 'Action on data' on tMysqlOutput, but nothing seems to fit (I don't really want to update existing data).
Any suggestion would be very appreciated.
Thanks and regards,
Massimo

Re: How to ignore 'duplicate key' errors

Anonymous — Thu, 10 Sep 2009 15:14:40 GMT

Either fetch unique values from the database or use the tUniqueRow component for moving unique rows to the target table.
Thanks and Regards,
Pravu Mishra.

Re: How to ignore 'duplicate key' errors

Anonymous — Thu, 10 Sep 2009 15:30:14 GMT

Try the action "insert if not exist".

Re: How to ignore 'duplicate key' errors

Anonymous — Thu, 10 Sep 2009 17:03:03 GMT

Hi nkxxDV,
unfortunately the only 'Action on data' I get in Component panel of tMysqlOutput are:
- insert
- update
- insert or update
- update or insert
- delete
- replace
- insert or update on duplicate key or unique index
I run TOS 3.1.3 build r26090.
Regards,
Massimo

Re: How to ignore 'duplicate key' errors

Anonymous — Thu, 10 Sep 2009 17:11:00 GMT

Ok, indeed it only exists for tMSSqlOutput.
Then I suggest you to fetch all the primary keys from your destination table and use it as a lookup for your main flow. You will do the join on the primary key and only output rejected rows.

Re: How to ignore 'duplicate key' errors

Anonymous — Thu, 10 Sep 2009 20:11:11 GMT

It seems to be working.
Thanks,
Massimo

Re: How to ignore 'duplicate key' errors

Anonymous — Tue, 05 Jun 2012 03:18:56 GMT

What about choosing insert or update on duplicate key or unique index AND Advanced Settings > Use Field Options and selecting the primary key column(s) as Updateable

edit: ETL seemed to work even without any updateable columns but this is invalid syntax and it would sometimes (?) recognise it or will hang so you are better to update the primary key (i.e. change nothing)