Incremental Load for the last 60 days data

[Anonymous]

Hi,
I'm a bit new to Talend and I have created some jobs to fetch data from APIs and posting into a Database. Now I need to modify the jobs where, when I run the job it should delete at 60 days of data and extract again and then merge with the historical data. Can anyone please help me to solve this issue.
Thanks in advance

[billimmer]

Use a global variable to hold the dates that you want to prune, and reload.  Then use the global variable in your "WHERE" statement in your SQL.  For example in a tjava component:

Date d_from;

Date d_to;

String s_from_date;

String s_to_date;

// get date range

d_to = TalendDate.getCurrentDate();

d_from = TalendDate.addDate(d_to,-60,"dd");

// convert to strings in the format of your data.  In this 

// example, dates are stored as YYYYMMDD in my DB.

s_to_date = TalendDate.formatDate("yyyyMMdd",d_to);

s_from_date = TalendDate.formatDate("yyyyMMdd",d_from);

// Store in global variable

globalMap.put("myToDate", s_to_date);

globalMap.put("myFromDate", s_from_date);

// show your work

System.out.println(" To Date: " + (String)globalMap.get("myToDate"));

System.out.println(" From Date: " + (String)globalMap.get("myFromDate"));

[Anonymous]

Thanks for the reply. Can you please elaborate more o doing this task. I meant the components which I have to use and how to configure them. I'm pretty new to Talend that's why

[billimmer]

You will be using the t(your_db)sqlRow component to prune.  To insert records you can use the t(your_db)Output or t(your_db)OutputBulkExec.

I recommend you read the on-line documentation on the database components for the database you are using.  In that documentation there is an example for each.

Also, there is an e-book on Amazon called "Talend Open Studio Cookbook".  This is a good book that will make you productive quickly.