MySQL CDC Capture - understanding logic

vapukov — Thu, 16 Mar 2017 08:36:03 GMT

Hi Team,
I never use Talend CDC for sync databases because prefer use other methods, but now one friend ask me to help and I meet some problem, which not expect.
So,
Connections - created, schemas created
Subscriber created, triggers - all ok
Next step create Job - capture "changes" and split for 3 flow - Insert/Update/Delete
also clean
Leave Job for sometime with regular scheduling ... and it crash by null pointer exception, let go investigate

All Inserts - work fine
All Updates - work also fine, and it work fine - if records was Updated twice - it catch only 1 Update, Perfect!!!
Deletes - as well, until ...

Until at the time between iteration same records was - Updated, and then Deleted
Talend not understand this situation and try to Update records with New data where all columns other than PK - NULLS and forget about NOT NULL constraints.
More - it crash not on UPDATE state, where it possible to manage, but crash on read state, because his own Metadata structure has same NOT NULL constraints.
I create testing Job for illustrate situation

Situation is realistic and my question is:
- what Talend think when design trigger based CDC without saving OLD and NEW data?
- or why in this case CDC not do the same as for UPDATE - fetch only delete if records deleted?

Re: MySQL CDC Capture - understanding logic

Anonymous — Fri, 13 Apr 2018 07:49:19 GMT

Hello ,

We are supposing that you have read the online document about:TalendHelpCenter:CDC architectural overview

Could you please clarify in which Talend version/edition you are?

Best regards

Sabrina

topic Re: MySQL CDC Capture - understanding logic in Talend Studio

MySQL CDC Capture - understanding logic

Re: MySQL CDC Capture - understanding logic