Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi Team,
I never use Talend CDC for sync databases because prefer use other methods, but now one friend ask me to help and I meet some problem, which not expect.
So,
Connections - created, schemas created
Subscriber created, triggers - all ok
Next step create Job - capture "changes" and split for 3 flow - Insert/Update/Delete
also clean
Leave Job for sometime with regular scheduling ... and it crash by null pointer exception, let go investigate
All Inserts - work fine
All Updates - work also fine, and it work fine - if records was Updated twice - it catch only 1 Update, Perfect!!!
Deletes - as well, until ...
Until at the time between iteration same records was - Updated, and then Deleted
Talend not understand this situation and try to Update records with New data where all columns other than PK - NULLS and forget about NOT NULL constraints.
More - it crash not on UPDATE state, where it possible to manage, but crash on read state, because his own Metadata structure has same NOT NULL constraints.
I create testing Job for illustrate situation
Situation is realistic and my question is:
- what Talend think when design trigger based CDC without saving OLD and NEW data?
- or why in this case CDC not do the same as for UPDATE - fetch only delete if records deleted?
I never use Talend CDC for sync databases because prefer use other methods, but now one friend ask me to help and I meet some problem, which not expect.
So,
Connections - created, schemas created
Subscriber created, triggers - all ok
Next step create Job - capture "changes" and split for 3 flow - Insert/Update/Delete
also clean
Leave Job for sometime with regular scheduling ... and it crash by null pointer exception, let go investigate
All Inserts - work fine
All Updates - work also fine, and it work fine - if records was Updated twice - it catch only 1 Update, Perfect!!!
Deletes - as well, until ...
Until at the time between iteration same records was - Updated, and then Deleted
Talend not understand this situation and try to Update records with New data where all columns other than PK - NULLS and forget about NOT NULL constraints.
More - it crash not on UPDATE state, where it possible to manage, but crash on read state, because his own Metadata structure has same NOT NULL constraints.
I create testing Job for illustrate situation
Situation is realistic and my question is:
- what Talend think when design trigger based CDC without saving OLD and NEW data?
- or why in this case CDC not do the same as for UPDATE - fetch only delete if records deleted?
