Java heap space talend

[Anonymous]

Hi All,

I have a simple job with the following components
tMySqlInput ---->tMap---->tMySqlOutput
This is for inserting/updating the records from 1 MySql Db to another.This is working fine for a table with 100 000 records .I am getting an error when tried with a table of 3 995 586 records

I check the option enable stream and I modify the Use specific JVM arguments In the Run view, open the Advanced Settings tab and in the file TOS_ESB-win-x86_64

but still dont work any solutions plzzzzzzzz

[Anonymous]

@AnisBensalah 

I am sorry. I did not get what you are trying to convey. Could you please explain the current issue with some screenshots?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[Anonymous]

hi @nthampi,

well i want to insert 4 million records  from DB to another DB for that i use  a simple job with the following components
tMySqlInput ---->tMap---->tMySqlOutput but when i start the job it remains running for an hour or two and it gives me only 200 record and finally no result ,and in the end therse no error, I do not know why  

 

[Anonymous · Accepted solution]

Hi,

    Could you please use Bulk components to load the data to target table? So you can download the data from source table and if you are using tparallelize, you can download multiple partitions of data from source table at same time.

     Once the data is ready, you can use Bulk components to load them to target table. Once you do this method, your overall processing time will be reduced from hours to minutes.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[Anonymous]

thnx @ nikhilthampi it s work

[sushantk19]

@AnisBensalah @nthampi _: I am also facing the same issue. I have a simple job design where i am extracting the data from source and loading to my target table( around 3 to 5 million). what do you mean by bulk components? how to parallelize it? my job design is as follows:

tMysqlinput--->tLogRow--->tMap-->tDBoutput

[sushantk19]

@AnisBensalah @nthampi _: I am also facing the same issue. I have a simple job design where i am extracting the data from source and loading to my target table( around 3 to 5 million). what do you mean by bulk components? how to parallelize it? my job design is as follows:

tMysqlinput--->tLogRow--->tMap-->tDBoutput