topic Re: slow talend in Talend Studio

slow talend

Anonymous — Sat, 16 Nov 2024 11:24:23 GMT

Hello,
I am using Talend BIg data.But i am facing problem now.Talend has become very slow now. I dont know why it has become like that.
Previously it was very fast.
I am trying to load 10000 rows but it taking 400seconds for that.
Could you please tell me what is the problem behind this???
Regards,
Vinuta

Re: slow talend

Anonymous — Mon, 01 Dec 2014 10:17:11 GMT

Hi,
Is it slow just for your specific job or the whole studio performance is worse than before?
Best regards
Sabrina

Re: slow talend

Anonymous — Mon, 01 Dec 2014 10:25:03 GMT

Hello,
The studio performance is worse than before.
If I load the job it is coming as GC overhead limit exceeded.
Regards,
Vinuta

Re: slow talend

Anonymous — Mon, 01 Dec 2014 11:03:36 GMT

hi all,
you haven't got enought memory to do the Job.
GC overhead is about the fact that JVM spent to many time to his garbage collector.
It's due to too many java object created for the RAM available.
Increase memory or (better way) refactoring your job to optimize data management.
regards
laurent

Re: slow talend

Anonymous — Mon, 01 Dec 2014 11:10:14 GMT

Hi,
can you please tell me how to do that??
Regards,
Vinuta

Re: slow talend

Anonymous — Mon, 01 Dec 2014 12:22:20 GMT

Hello,
I have installed java environment 1.7.0_71 version for windows 8.1. Can i use this version for Talend Bigdata???
Vinuta

Re: slow talend

Anonymous — Mon, 01 Dec 2014 13:00:54 GMT

hello,
I have a job that looks like:
tredshiftinput > tMap >> tmysqlOutput
My file has million rows and I get to loaded I get the error. What can I do differently to avoid this Java heap space error?
Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space
at java.util.LinkedList.listIterator(Unknown Source)
at java.util.AbstractList.listIterator(Unknown Source)
at java.util.AbstractSequentialList.iterator(Unknown Source)
at routines.system.RunStat.sendMessages(RunStat.java:261)
at routines.system.RunStat.run(RunStat.java:225)
at java.lang.Thread.run(Unknown Source)
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Unknown Source)
at java.lang.Class.getDeclaredField(Unknown Source)
at java.util.concurrent.atomic.AtomicReferenceFieldUpdater$AtomicReferenceFieldUpdaterImpl.<init>(Unknown Source)
at java.util.concurrent.atomic.AtomicReferenceFieldUpdater.newUpdater(Unknown Source)
at java.sql.SQLException.<clinit>(Unknown Source)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1811)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
disconnected
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:372)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:252)
at project_1.r_m_refr_0_1.r_m_refr.tRedshiftInput_1Process(r_m_refr.java:1046)
at project_1.r_m_refr_0_1.r_m_refr.runJobInTOS(r_m_refr.java:1800)
at project_1.r_m_refr_0_1.r_m_refr.main(r_m_refr.java:1657)
Regards,
Vinuta

Re: slow talend

Anonymous — Tue, 02 Dec 2014 07:28:05 GMT

Hi,

Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space

Have you already checked KB articles:https://help.talend.com/pages/viewpage.action?pageId=190513217 and https://community.talend.com/t5/Migration-Configuration-and/OutOfMemory-Exception/ta-p/21669?content-lang=en ?

Best regards
Sabrina

Re: slow talend

Anonymous — Tue, 02 Dec 2014 08:06:17 GMT

Hi,

I have installed java environment 1.7.0_71 version for windows 8.1. Can i use this version for Talend Bigdata???

Please check document about: TalendHelpCenter:Compatible Platforms
Best regards
Sabrina

Re: slow talend

Anonymous — Tue, 02 Dec 2014 09:23:15 GMT

Hi all,
I have checked these blogs. For windows 8.1 64 bit operating system oracle java 7 is supported.so, I have installes java 1.7.1_71 version.
But If I try to check the version and incresing memory in command prompt it is coming as java version 1.8 is required. I do not know whether 1.8 is compatible for Talend big data??
What should I do now??Can you please tell me??
Regards,
Vinuta

Re: slow talend

Anonymous — Tue, 02 Dec 2014 09:28:02 GMT

Hi,
So far, talend don't support for Java 8. TalendHelpCenter:Java 8 Support
Could you please follow up document TalendHelpCenter:Exception: outOfMemory to allocate more memory for your job?

Best regards
Sabrina

Re: slow talend

Anonymous — Tue, 02 Dec 2014 09:53:44 GMT

Hi,
I do not know about job script. Can I allot 4096M memory for oracle java 7. is that supports??
I am using talend bigdata version 5.6. is that any problem with this version??
Regards,
Vinuta

Re: slow talend

Anonymous — Tue, 02 Dec 2014 11:59:03 GMT

hello,
I increased the memory also.but i have not done optimization since I am using only one file.
But still its not loading. It ll come as starting but rows are not loading.
I do not know what is the problem.
Talend big data version : 5.6
java version : 1.7.0_71
windows 8.1- 64 bit operating system
Could you please tell me????
Regards,
Vinuta

Re: slow talend

Anonymous — Wed, 03 Dec 2014 04:06:02 GMT

Hi,
Is there any error info printed on console? Is it still " java.lang.OutOfMemoryError: Java heap space"? Or the error which is mentioned in your another topic: https://community.talend.com/t5/Design-and-Development/resolved-load-data-from-amazon-redshift-to-mysql/td-p/71004?
Best regards
Sabrina

Re: slow talend

Anonymous — Wed, 03 Dec 2014 06:56:32 GMT

Hi Vinuta,
- Are you using any lookup in tMap? if yes, you can opt for using FileSystem for lookup data storage in place of memory...
- Whether your output DB is on same machine where redshift is available? if not, then network delay could be an issue
- What are other processes are consuming memory when you execute the talend job and how much memory is available for talend?
- What is the batchsize in output, try tweaking that..
- Have you tried restarting Talend?
Vaibhav