https://community.qlik.com/t5/Talend-Studio/Performance-of-running-deduplication-using-tMatchGroup-and/m-p/2202978#M4483 <P>Hi,</P> <P>Is anyone using tMatchGroup to deduplicate customer master data on Talend?</P> <P>I am currently using Swoosh on top of a 2M rows dataset and I am constantly getting out of memory errors...</P> <P>My server is an Azure VM with 16GB of RAM and 4vCores that are used pretty much at 100% through the whole processing time.</P> <P>&nbsp;</P> <P>I am wondering why the performance is so poor, and how I will be able to scale it since I have 13 sources yet to be added to the same dataset...</P> <P>&nbsp;</P> <P>Any suggestions and considerations?</P> <P>How to size and estimate the processing requirements for this task?</P> <P>What can be tuned in terms of settings of the Run or Component?</P> <P>&nbsp;</P> <P>Thank you</P> <P><BR />Regards,</P> <P>LS</P> Sat, 16 Nov 2024 03:15:29 GMT Luis_Simoes 2024-11-16T03:15:29Z https://community.qlik.com/t5/Talend-Studio/Performance-of-running-deduplication-using-tMatchGroup-and/m-p/2202978#M4483 <P>Hi,</P> <P>Is anyone using tMatchGroup to deduplicate customer master data on Talend?</P> <P>I am currently using Swoosh on top of a 2M rows dataset and I am constantly getting out of memory errors...</P> <P>My server is an Azure VM with 16GB of RAM and 4vCores that are used pretty much at 100% through the whole processing time.</P> <P>&nbsp;</P> <P>I am wondering why the performance is so poor, and how I will be able to scale it since I have 13 sources yet to be added to the same dataset...</P> <P>&nbsp;</P> <P>Any suggestions and considerations?</P> <P>How to size and estimate the processing requirements for this task?</P> <P>What can be tuned in terms of settings of the Run or Component?</P> <P>&nbsp;</P> <P>Thank you</P> <P><BR />Regards,</P> <P>LS</P> Sat, 16 Nov 2024 03:15:29 GMT https://community.qlik.com/t5/Talend-Studio/Performance-of-running-deduplication-using-tMatchGroup-and/m-p/2202978#M4483 Luis_Simoes 2024-11-16T03:15:29Z