output data for 1st load: id,name,created_date,modified_date 1,ramu,18-07-2019,18-07-2019 2,rah,18-07-2019,18-07-2019 3,afaAaa,18-07-2019,18-07-2019
My input data have some modifications(1 row update and 2 rows insert) as below
id,name 1,ramu 2,rah 3,afa 4,SADASE 5,akda
So output should be as below: id,name,created_date,modified_date 1,ramu,18-07-2019,18-07-2019 2,rah,18-07-2019,18-07-2019 3,afa,18-07-2019,19-07-2019 4,SADASE,19-07-2019,19-07-2019 5,akda,19-07-2019,19-07-2019
The rows which are not changed/updated should not have in the change of modified_date. I have done this by taking lookup of output table with input file which is taking very long time. Can you please help on alternate solution
Since you already know the ids of the lookup data, do not pull the entire data of the table in lookup. First do an insert of new data to a temp table. For the lookup, do an inner join of the id of temp table with your lookup table so that you are picking only matched records instead of full table.
You can do it my adding join conditions in the SQL query of tDBinput for lookup. In the tMap join, use left out join for the main flow with your new lookup and add the condition that if the data id is present in lookup, use the date from lookup else add current system date. This should resolve your issue. If the issue persists, please share the screenshot with these changes and error and we will take it forward from there.
Warm Regards, Nikhil Thampi
Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂
What is the input volume and how much time its currently taking for the query? Could you please share the execution plan details if the query is taking more time?
Warm Regards, Nikhil Thampi
Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂
