topic Re: Best Practice for insert and updates in Talend Studio

Best Practice for insert and updates

Anonymous — Thu, 14 Feb 2019 21:12:48 GMT

I have good solution for insert and updates.
(With this solution you can perform inserts and updates very very quickly)
When you are getting records from the source from any data base, use tmap after any inputDb and do look up with the target , and use inner join lookup model with the cloumn to be matched, Then whatever it matches with key store all records in Temp table, Create table in the job run time.The other new which does not match with the key will go to direct insert.
Than using Merge concept sync temp table and target
After that delete temp table at the end
Example of Merge query (By using this merge query it will sync source and target with in seconds)

MERGE Table A AS TARGET
USING Table B AS SOURCE
ON TARGET.tableacol1= SOURCE.col1
WHEN MATCHED
THEN
UPDATE
SET

TARGET.col2= SOURCE.col2,
TARGET.col3=SOURCE.col3;

Re: Best Practice for insert and updates

Vinisha — Thu, 14 Feb 2019 21:16:33 GMT

yes it worked, it improved the speed of the job , rather using insert/update inbuilt, its better to use, when we have Million of records

Re: Best Practice for insert and updates

vapukov — Fri, 15 Feb 2019 05:39:22 GMT

@Vinisha wrote:

yes it worked, it improved the speed of the job , rather using insert/update inbuilt, its better to use, when we have Million of records

and it kill your Talend server when you have 100 millions and what you will do when you have a billion?

Re: Best Practice for insert and updates

Anonymous — Mon, 12 Aug 2019 13:26:58 GMT

Hi,

Can you please explain in more details.

Thanks,