I'm working with talend to transform data from oracle database to postgresql database but i have a problem with the time of charging data, it takes too much time so is there any solution for this problem
the output of the tmap componant is the one which slow my job , i have to move almost 3.000.000 lines from the source to the target . the componants which i use in my job are postgresqlinput oracleinput , postgresqloutput tmap and tunit .
Hi,
In most cases, tMap is not the cause of slow process and 3,000,000 rows is not a monster.
Did you use statistics to measure response time for each component?
Did you measure select duration individualy? Are they simple or complex queries?
In my opinion, you should focus on the Postgresql part and try to bulkify the transaction because of the number of rows. See tPostgresqlOutputBulkExec for this purpose.
