topic Re: the time of charging data in Talend Studio

the time of charging data

Anonymous — Tue, 25 Apr 2017 09:28:29 GMT

I'm working with talend to transform data from oracle database to postgresql database but i have a problem with the time of charging data, it takes too much time so is there any solution for this problem

Re: the time of charging data

TRF — Tue, 25 Apr 2017 10:36:27 GMT

Can you detail your job? Which components are using? Which one seems to slow your job? How many rows are you trying to move from source to target?

Re: the time of charging data

Anonymous — Wed, 26 Apr 2017 09:40:45 GMT

the output of the tmap componant is the one which slow my job , i have to move almost 3.000.000 lines from the source to the target . the componants which i use in my job are postgresqlinput oracleinput , postgresqloutput tmap and tunit .

Re: the time of charging data

TRF — Wed, 26 Apr 2017 21:28:32 GMT

Hi,
In most cases, tMap is not the cause of slow process and 3,000,000 rows is not a monster.
Did you use statistics to measure response time for each component?
Did you measure select duration individualy? Are they simple or complex queries?
In my opinion, you should focus on the Postgresql part and try to bulkify the transaction because of the number of rows. See tPostgresqlOutputBulkExec for this purpose.

Let us know.