Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi. I working on a TO S.
My Job project has lots of Jobs.(below image)
You should look at the left image in a "FRONTDOM_18" Folder, it has 36 Jobs(child job), 2 Starter(Parent job)
and each job is configured below image.
[ setting : Multi thread execution(ON), db connection(shared), db commit(on), db close(off) in the child job)
and starter ( parent job )
[ setting : Multi thread execution(ON), db connection(shared), db close(on) in the child job, Xms 2G, Xmx 4G)
Q, There is no problem when running separately. however, errors occur only when they are bind and executed.
I think It's obviously a problem with memory, but i don't know how deal with this problem.
Is there anything else besides increasing memory?
Q, If solution exists, can i run the Jobs Folder List(same time) on the first image
sorry for bad english skill;
thanks.
Exception in thread "Thread-122" java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.Arrays.copyOf(Unknown Source) at java.lang.StringCoding.safeTrim(Unknown Source) at java.lang.StringCoding.access$300(Unknown Source) at java.lang.StringCoding$StringEncoder.encode(Unknown Source) at java.lang.StringCoding.encode(Unknown Source) at java.lang.String.getBytes(Unknown Source) at org.postgresql.core.Utils.encodeUTF8(Utils.java:54) at org.postgresql.core.v3.SimpleParameterList.getV3Length(SimpleParameterList.java:234) at org.postgresql.core.v3.QueryExecutorImpl.sendBind(QueryExecutorImpl.java:808) at org.postgresql.core.v3.QueryExecutorImpl.sendOneQuery(QueryExecutorImpl.java:1053) at org.postgresql.core.v3.QueryExecutorImpl.sendQuery(QueryExecutorImpl.java:644) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:345) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2674) at cdc.frontdom_193_198_0_1.FRONTDOM_193_198.tPostgresPlusInput_3Process(FRONTDOM_193_198.java:7841) at cdc.frontdom_193_198_0_1.FRONTDOM_193_198$4.run(FRONTDOM_193_198.java:15021) Batch entry 43 <unknown> was aborted. Call getNextException to see the cause. 18_130 : 43 Exception in component tPostgresPlusInput_6 (FRONTDOM_193_198) org.postgresql.util.PSQLException: Ran out of memory retrieving query results. at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1309) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:192) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:451) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:336) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:235) at cdc.frontdom_193_198_0_1.FRONTDOM_193_198.tPostgresPlusInput_6Process(FRONTDOM_193_198.java:14041) at cdc.frontdom_193_198_0_1.FRONTDOM_193_198$7.run(FRONTDOM_193_198.java:15153) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.postgresql.core.PGStream.ReceiveTupleV3(PGStream.java:349) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1306) ... 6 more 18_160 : 296 18_192 : 661 18_180 : 17 18_125 : 171
Exception in component tRunJob_273 (FRONTDOM_STARTER_2) java.lang.RuntimeException: Child job running failed. org.postgresql.util.PSQLException: Ran out of memory retrieving query results. at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1309) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:192) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:451) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:336) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:235) at cdc.frontdom_193_198_0_1.FRONTDOM_193_198.tPostgresPlusInput_6Process(FRONTDOM_193_198.java:14041) at cdc.frontdom_193_198_0_1.FRONTDOM_193_198$7.run(FRONTDOM_193_198.java:15153) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.postgresql.core.PGStream.ReceiveTupleV3(PGStream.java:349) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1306) ... 6 more at cdc.frontdom_starter_2_0_1.FRONTDOM_STARTER_2.tRunJob_273Process(FRONTDOM_STARTER_2.java:4792) at cdc.frontdom_starter_2_0_1.FRONTDOM_STARTER_2$15.run(FRONTDOM_STARTER_2.java:6436) 18_186 : 532041 FRONTDOM_Stater is completed 521088 milliseconds
Thank to reply,
but, I search the error message from my question,
I found it UseGCOverheadLimit option is just error ignore(masking)..
I think preparedstatemts cache size.
Anyway, I solved the problem with Batch Size = 10000 to 5000, exec time little bit slow, but error does not appears,
Thank to reply
Hi
please try the Following workaround to solved the problem in Talend
Add new option to Windows–>Preferences–>Talend–>Run/Debug – XX:-UseGCOverheadLimit
Thank to reply,
but, I search the error message from my question,
I found it UseGCOverheadLimit option is just error ignore(masking)..
I think preparedstatemts cache size.
Anyway, I solved the problem with Batch Size = 10000 to 5000, exec time little bit slow, but error does not appears,
Thank to reply