Incremental data load from Oracle database to PostgreSQL database

[Anonymous]

Hi,
I want to implement an Incremental data load using Talend. how to identify incremental data using the updated_date column in the oracle server & how to load incremental data into the PostgreSQL server using Talend.  how to process incremental data CDC (change data capture) logic using Talend. 

[manodwhb]

@srinivask , check the bellow link.

https://community.talend.com/t5/Design-and-Development/Incremental-Load/td-p/157128

[Anonymous]

Hi Manohar,

Thanks for the given details.

1) Approach 1: how to store the last run date? where do I need to store the last_update column value & job details? In my case source is OracleDB & target PostgreSQL.  could you plz provide any example. 

2)  Approach 2: how to fetch the last run date (or) max(load_date) from the target database. how to use "context.lastrun" variable value? could you plz provide any example.

[manodwhb]

@srinivask , 1) Last_Rundate that you can store on a file or table. which you need to take the current date from Oracle DB and nee to load.

2) what even you have loaded Last_Rundate table that you need to read using tDBinput and set as a global variable or assigned as context variable.

[Anonymous]

Hi 

The approach I understand.  I'm new to Talend ETL. could you please provide an example.  
1) we don't have access to create a new table or file in Oracle for last_rundate. I need to upload it to PostgreSQL only. How to fetch the current date from OracleDB?  
2) How to create a context variable and fetch context variable value? 

[manodwhb]

@srinivask , create a table with last_rundate in postgresql. and read that table to get the last_rundate.

using the below design you wil get the lastrundate.

tDBInput(Oracle)-->tDBoutput(postgressql)

query should be like SELECT TO_CHAR(SYSDATE, 'DD-MON-YYYY HH:MIS') FROM dual