topic how to get only modified data or changed data from mysql using talend in Talend Studio

how to get only modified data or changed data from mysql using talend

Anonymous — Sat, 16 Nov 2024 09:25:25 GMT

Hi ,

Can anyone please explain how can I get only changed data.

I have read somewhere that if we apply CDC, we can get only changed or modified data but the CDC is not applicable in open source.

So, I have run one job in talend tool, output I have placed in MySQL.

if I do SCD , I will get both history and modified data.

but I want only changed data in my output.

what can I do?

please anyone help me in this...

Re: how to get only modified data or changed data from mysql using talend

Anonymous — Thu, 10 Aug 2017 12:45:40 GMT

If you store your data in a SCD you can query for only changed data. Otherwise you will have to keep a "change date" field on your table (or linked to your table's primary key) and refer to that.

Re: how to get only modified data or changed data from mysql using talend

Anonymous — Thu, 10 Aug 2017 13:41:56 GMT

please could you explain me clearly

Re: how to get only modified data or changed data from mysql using talend

Anonymous — Thu, 10 Aug 2017 14:22:15 GMT

With a slowly changing dimension you should be able to easily query recently modified records. All you would need to do is keep a log (in a different table) of the last time you had run your job and then use the effective dating in the SCD with your last run date.

Alternatively add a change_date column to your database table and update it when you make a change to a row