Talend job freezes during Oracle data extraction

[Anonymous]

Hello Talend's community !

I'm currently using a Talend job that extract data from Oracle database and insert them into a SnowFlake database. I'm using a loop to extract data day by day (see screenshot in attachment). I'm doing that to relieve the source database.

My problem is that from time to time (it happens randomly) the Talend job is freezing, the insertion flow is stuck but there is no error message and the job is still "running" even if nothing happen.

Do you think that the Oracle database is the problem ? And what can I do to avoid / fix this problem ? as I said I have no error messages (not even a timeout from the database).

Thank you for your help,

Best regards,

Thomas

[Anonymous]

Hi,

    Are you running the Talend job in a Job server? If yes, you may have to verify whether its having recommended sizing.

A high level thumb rule is as shown above but it also depends on the data volume you would like to process.

Now, by any chance are you running the Talend job from your own system and monitor the process real time? It may go to freeze mode when its not getting enough computational resources from your system. When you are using your desktop/laptop, multiple other processes may be running in parallel and it may not have enough system resources to process a heavy data load. 

So if you are running the job in your laptop for testing purpose, make sure that you are giving enough CPU, memory, disk space and even a good network bandwidth.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

[Anonymous]

Hello Nikhil  and thank you for your answer !

I'm running my Talend job on a Windows AWS virtual machine with 32Go of RAM and 8CPU, I allocated around 10GO of RAM to my job and it seem's to be enough. My disk space is also big enough, maybe the problem comes from the network bandwidth, I looked at the monitoring console during the job process and I saw that the TCP connection was saturated, maybe this is the issue....

Moreover I'm not the owner of the Oracle database that I'm trying to extract, maybe my access to the database is limited...

Thank you for those clues !

Regards,

Thomas

[Anonymous · Accepted solution]

@tom77380 

Hi Thomas,

     Thanks for the update. Most probably the issue could be due to network saturation in your case. 

     Could you please mark the topic as resolved if there are no further queries?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved