Skip to main content
Announcements
Join us at Qlik Connect for 3 magical days of learning, networking,and inspiration! REGISTER TODAY and save!
cancel
Showing results for 
Search instead for 
Did you mean: 
VenkyM
Contributor
Contributor

MySQL CDC functionality is not working

Hi,

I am trying to learn how the Change Data Capture (CDC) works.

As part of it, I followed the steps given in the following link.

https://help.talend.com/reader/RLLVjeyBol_aw8PB0tyxJg/4Te87GKIHEWPvhnK0YY3xQ

My source and target are MYSQL. One is on the remote server, which is my source. The second one is my target, which in on my local laptop.

Created the connections as per the step1

0693p000008vYjIAAU.png

While executing the script created during "Add CDC" step, i got the below error. I ignore the errors and proceeded.

0693p000008vYd0AAE.png

I tried to test it by inserting a row into the source table, which is customer. The row got inserted, but i did not see any data in my target table. My understanding was that when I insert a row into the source, the triggers will automatically push the changes to the target table, which is on my local mysql.

The below is the screenshot of the source table.

0693p000008vYldAAE.png

The view all changes is empty.

0693p000008vYW0AAM.png

I am guessing that the connection was not established properly. Any help will be appreicated!

Labels (3)
3 Replies
VenkyM
Contributor
Contributor
Author

Here is the script generated by the add cdc operation.

 

 

alter table `sakila`.`TSUBSCRIBERS` modify `TALEND_CDC_TABLE_TO_WATCH` VARCHAR(255)

 

INSERT INTO `sakila`.`TSUBSCRIBERS`

 (

  `TALEND_CDC_TABLE_TO_WATCH`,

  `TALEND_CDC_SUBSCRIBER_NAME`,

  `TALEND_CDC_CREATION_DATE`

 )

  values ('entries1.customer','APP1',sysdate()) ;

 

 

CREATE TABLE `sakila`.`TCDC_customer` 

 ( 

  `TALEND_CDC_SUBSCRIBERS_NAME` VARCHAR(50) NOT NULL, 

  `TALEND_CDC_STATE` VARCHAR(1), 

  `TALEND_CDC_TYPE` VARCHAR(1), 

  `TALEND_CDC_CREATION_DATE` DATETIME,

  `customerId` INT(10) NOT NULL

 ) ;

 

 

CREATE OR REPLACE VIEW `sakila`.`TCDC_VIEW_customer` AS 

 SELECT

 SRC.`TALEND_CDC_TYPE` TALEND_CDC_TYPE,

 SRC.`TALEND_CDC_SUBSCRIBERS_NAME` TALEND_CDC_SUBSCRIBERS_NAME,

 max(SRC.`TALEND_CDC_CREATION_DATE`) TALEND_CDC_CREATION_DATE,

 SRC.`customerId` customerId,

 TARG.`name` name,

 TARG.`countryOfResidence` countryOfResidence

 FROM `sakila`.`TCDC_customer` SRC LEFT JOIN `entries1`.`customer` TARG  

 ON SRC.`customerId`=TARG.`customerId`

 WHERE SRC.`TALEND_CDC_STATE`='1'

 GROUP BY SRC.`TALEND_CDC_SUBSCRIBERS_NAME`, SRC.`customerId`,

  SRC.`TALEND_CDC_TYPE` ;

 

 

CREATE TRIGGER `entries1`.`TCDC_TG_customer_I` 

 AFTER INSERT ON `entries1`.`customer`

 FOR each row 

 INSERT INTO `sakila`.`TCDC_customer` 

 ( 

  `TALEND_CDC_SUBSCRIBERS_NAME`,

  `TALEND_CDC_STATE`,

  `TALEND_CDC_TYPE`,

  `TALEND_CDC_CREATION_DATE`,

  `customerId`

 ) SELECT 

   `TALEND_CDC_SUBSCRIBER_NAME`,

   '0',

   'I',

   sysdate(),

   new.`customerId`    

  FROM `sakila`.`TSUBSCRIBERS`

  WHERE `TALEND_CDC_TABLE_TO_WATCH`='entries1.customer' ;

 

CREATE TRIGGER `entries1`.`TCDC_TG_customer_U` 

 AFTER UPDATE ON `entries1`.`customer`

 FOR each row 

 INSERT INTO `sakila`.`TCDC_customer` 

 ( 

  `TALEND_CDC_SUBSCRIBERS_NAME`,

  `TALEND_CDC_STATE`,

  `TALEND_CDC_TYPE`,

  `TALEND_CDC_CREATION_DATE`,

  `customerId`

 ) SELECT 

   `TALEND_CDC_SUBSCRIBER_NAME`,

   '0',

   'U',

   sysdate(),

   new.`customerId`

  FROM `sakila`.`TSUBSCRIBERS`

  WHERE `TALEND_CDC_TABLE_TO_WATCH`='entries1.customer' ;

 

CREATE TRIGGER `entries1`.`TCDC_TG_customer_D` 

 AFTER DELETE ON `entries1`.`customer`

 FOR each row 

 INSERT INTO `sakila`.`TCDC_customer` 

 ( 

  `TALEND_CDC_SUBSCRIBERS_NAME`,

  `TALEND_CDC_STATE`,

  `TALEND_CDC_TYPE`,

  `TALEND_CDC_CREATION_DATE`,

  `customerId`

 ) SELECT 

   `TALEND_CDC_SUBSCRIBER_NAME`,

   '0',

   'D',

   sysdate(),

   old.`customerId` 

  FROM `sakila`.`TSUBSCRIBERS`

  WHERE `TALEND_CDC_TABLE_TO_WATCH`='entries1.customer' ;

 

Anonymous
Not applicable

Hi

 "values ('entries1.customer','APP1',sysdate()) ;"

About the error 'unknown database entries1'

The database used for creating CDC tables and the database which contains the source table should be in the same Mysql server, I guess the database used for creating CDC tables is in your local Mysql.

 

Regards

Shong

VenkyM
Contributor
Contributor
Author

Thanks for the reply. Appreciated!

 

Let me check it.