Java heap space talend

[Anonymous]

Hi All,

I have a simple job with the following components
tMySqlInput ---->tMap---->tMySqlOutput
This is for inserting/updating the records from 1 MySql Db to another.This is working fine for a table with 100 000 records .I am getting an error when tried with a table of 3 995 586 records

I check the option enable stream and I modify the Use specific JVM arguments In the Run view, open the Advanced Settings tab and in the file TOS_ESB-win-x86_64

but still dont work any solutions plzzzzzzzz

[manodwhb]

@AnisBensalah ,can you please let me know what JVM configuration are you using? And can you also do the operation of tMap on store on disk.

[Anonymous]

Hi,

    You can increase the heap space for specific jobs or for all the jobs by changing the Xms and Xmx parameters.

https://community.talend.com/t5/Design-and-Development/Pass-JVM-arguments-in-quot-Use-specific-JVM-a...

    But it is not a single solution that fits all sizes. You need to still think about the possibility of parking the data in tMap and other components using temporary disk space. Please refer the detail below from below link.

https://help.talend.com/reader/EJfmjmfWqXUp5sadUwoGBA/J4xg5kxhK1afr7i7rFA65w

   It means that you are not holding the entire data in memory which usually creates the issue like in your use. It will hold the overflowing data in disk temporarily and this way you will not run out of memory and you dont have to keep on expanding the memory.

    I would also suggest you to do a memory run of the job to optimize the memory allocation of your job before moving to production. In this way, you will know what is the optimum value by changing the Xms and Xmx parameters.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[Anonymous]

hi @nthampi ,@manodwhb  thnx for repling me but it s the same probleme 

in fact there is no error but the job it stays in loop and does not end since 2 hours of the execution of the job what am I to do

---

---

[Anonymous]

Hi,

     Did you check at which component, it is taking more time to process? Is there any DB load as part of your job? Most probably it could be due to low commit size.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[Anonymous]

hi 

 there is no loading from the base and there is no transformation to the new base

I think that it stops at the levels of the loading . in the tdbinput 

[Anonymous]

Hi,

    I feel the tDBOutput is taking more time. Could you please change the highlighted parameters and see the performance difference. Please reach an optimum level for these parameters based on your system resource availability.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[Anonymous]

hi 

 the problem is not resolved but she gave me another error

Exception in component tDBOutput_1 (g)
java.sql.SQLException: Could not retrieve transation read-only status server

[Anonymous]

Hi,

    It seems multiple write requests without commit is happening at same time.

https://stackoverflow.com/questions/21651671/odd-sqlexception-could-not-retrieve-transation-read-onl...

   Could you please check whether commit is correct?

    I also saw another post in stackoverflow as below. Please think about this option too.

https://stackoverflow.com/questions/51935611/connecting-talnd-to-mysql-transation-read-only-status-s...

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[Anonymous]

hi @nthampi ,

I resolved the 2nd error its just it  an error in the connection parameter

but for the load data no 

I changed the number of commit every and the number of rows but still the same 0 ligne effected in the DB