topic Re: twritejsonfield performance issue in Talend Studio

twritejsonfield performance issue

SPal — Sat, 16 Nov 2024 03:18:13 GMT

Hello All,

I Need help in improving the performance of "twritejsonfield" component, currently this is processing around 2 to 3 rows per second which is too slow.

Please let me know if there is any solution or any alternate methods to increase the performance of converting records into Json file.

Regards,

Subrata

Re: twritejsonfield performance issue

TRF — Tue, 11 Feb 2020 12:40:36 GMT

For a huge json file with lany "parts", I suggest you to deal separatly with each part giving separated global variables.
At the end, concatenate thèse global variables to obtain the final result.
I use this pattern to build complex and large xml files and it works fine.

Re: twritejsonfield performance issue

SPal — Tue, 11 Feb 2020 13:39:54 GMT

Hi @TRF ,

Thanks for explaining this solution, In my scenario I have to create an array also and group the records based on incoming Id's is this some thing possible with global variables?. sorry for not explaining my scenario clearly.

Re: twritejsonfield performance issue

TRF — Wed, 12 Feb 2020 12:41:12 GMT

Probably yes.
I suggest you to add a sample to your post (what you have, what you want)