Delta load with Talend

[Anonymous]

Hi, 

I need to use delta load in my ETL. it is extracting from Mysql datasource and loading into Postgresql database. Mysql datasource has created_at and updated_at timestamps which I would like to use to extract new or update data. I have already implemented this in Sql Server with SSIS before. I am not sure how to implement with Talend.

Does anyone know implementing delta load with timestamps using Talend?

Thanks in advance.

Saanvi S

[brama · Accepted solution]

Hi,
As you have the date filed to identify the delta, it wil be easy.
Have two files one to have the current date(when the job flow starts). Another file last run date with low date 19000101. In the first load run the job and read date from last run date file, have this as a where clause to check source data timestamp col>run date and run the job. Next move the current date file value to next run date file. In the incremental run again the same process. So you will get the delta records.

Thanks,
Ramachandran B