unused data in update

mani1304 — Sat, 16 Nov 2024 04:48:35 GMT

Hi All,

My requirement as below.

I have a file, which will update the output table on the basis of key columns and conditions. Now, the requirement is to get the unused records of the file, which will not update the table.

One way is to do the inner join between file and table and fetch the reject records, but this is creating huge performance issue, because the table is around 30 million and it is affecting hugely.

Can someone suggest how to do it without degrading performance.

Re: unused data in update

TRF — Mon, 02 Sep 2019 07:31:03 GMT

If you use the option "Reload at each row" on tMap you may get rows from table 1 after 1 for each record from input file. Should be acceptable if input file is not too large.
However what if you don't filter anything and place tMap after your DbOutput component to catch updated record? (not able to try at this time)

topic unused data in update in Talend Studio

unused data in update

Re: unused data in update