Dear friends, I have a problem in Talend Open Studio. My job is described as below:
tOracleInput --->
(1) TMap (Modify data) -->
(2) tWriteJSONFields -->
(3) tMongoDBOutput
In Oracle database, there are around 300k records. The processing speed of (1) and (2) is around 3054,98rows/s, but in (3) is only around 10rows/s due to the process of tWriteJSONFields. The questions are:
- How can I improve the speed of (3)?
- Some time, there is an OutMemory Exception because data which is cached in (2) waiting for the process of tWriteJSONFields is over load. So can I limit a number of rows each time. More clearly, there are 300k records in Oracle, if the number of rows processed in tMap equals to an amount of rows (for example: for each 1000 records), the tOracleInput sleeps and wait for the next iteration.
Any suggestion will be appreciate. Thanks so much!
