topic Re: Need to update date if row changes in file in Talend Studio

Need to update date if row changes in file

Anonymous — Sat, 16 Nov 2024 05:13:34 GMT

Hi,

below is my input data:

id,name
1,ramu
2,rah
3,afaAaa

output data for 1st load:
id,name,created_date,modified_date
1,ramu,18-07-2019,18-07-2019
2,rah,18-07-2019,18-07-2019
3,afaAaa,18-07-2019,18-07-2019

My input data have some modifications(1 row update and 2 rows insert) as below

id,name
1,ramu
2,rah
3,afa
4,SADASE
5,akda

So output should be as below:
id,name,created_date,modified_date
1,ramu,18-07-2019,18-07-2019
2,rah,18-07-2019,18-07-2019
3,afa,18-07-2019,19-07-2019
4,SADASE,19-07-2019,19-07-2019
5,akda,19-07-2019,19-07-2019

The rows which are not changed/updated should not have in the change of modified_date.
I have done this by taking lookup of output table with input file which is taking very long time.
Can you please help on alternate solution

Thanks,

Ram

Re: Need to update date if row changes in file

Anonymous — Fri, 19 Jul 2019 18:16:04 GMT

Hi Ram,

Since you already know the ids of the lookup data, do not pull the entire data of the table in lookup. First do an insert of new data to a temp table. For the lookup, do an inner join of the id of temp table with your lookup table so that you are picking only matched records instead of full table.

You can do it my adding join conditions in the SQL query of tDBinput for lookup. In the tMap join, use left out join for the main flow with your new lookup and add the condition that if the data id is present in lookup, use the date from lookup else add current system date. This should resolve your issue. If the issue persists, please share the screenshot with these changes and error and we will take it forward from there.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Need to update date if row changes in file

Anonymous — Fri, 19 Jul 2019 18:20:19 GMT

Hi Nikhil,
Thanks for your reply.
As said I already Donne that thing,which will take more time in joining if records are more.

Can you please provide any alternate solution that only modified date will only change if old records get updated

Thanks,
Ram

Re: Need to update date if row changes in file

Anonymous — Fri, 19 Jul 2019 18:49:52 GMT

Hi,

What is the input volume and how much time its currently taking for the query? Could you please share the execution plan details if the query is taking more time?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂