I'm getting the following exception while running my job. The job is simple... it takes two inputs (from tPostgreSQLInput components) through a tMap. The result is stored as a delimted TXT file. Where should I be looking to rectify this? Our admin has stated that he has already given the Java VM the max allocation of memory.
### Job STARTED at 2009/01/22 12:13:25 (jobId=lrpusmu3, jobExecutionId=2hSY6uaM) ###
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.lang.String.replace(String.java:1879)
at java.net.URLClassLoader$1.run(URLClassLoader.java:191)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:299)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1309)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:192)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:451)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:336)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:235)
at dataextraction.partmaster_0_2.PartMaster.tPostgresqlInput_1Process(PartMaster.java:3361)
at dataextraction.partmaster_0_2.PartMaster.runJobInTOS(PartMaster.java:5611)
at dataextraction.partmaster_0_2.PartMaster.main(PartMaster.java:5491)
### Job ENDED WITH ERROR at 2009/01/22 12:14:51 (jobId=lrpusmu3, jobExecutionId=2hSY6uaM) ###
you might try using the "Enable stream" option on your main row to your tmap. This can be found under "Advanced settings".
Lookup tables will need to be loaded into memory.
I may be missing something, but my MAIN row tPostgreSQLInput component only has the settings: "Use Cursor", "Trim all the String/Char columns", and "tStatCatcher Statistics" options under Advanced Options. I'm using version 2.4.1. Was the Enable Stream option introduced after that?
"Enable stream" is the MySQL equivalent of Postgres "Enable Cursor"
Checking this option in your component should help you to reduce memory consuption on the main input.
This is not the only solution but it can be done as a first try.
Inyour lookup do you load all the columns of your table or did you select only the really required columns ?
HTH,
What would the other things I can try to eliminate this exception? Here's what we've done thusfar:
1) Allocated 2GB of memory to the Java VM
2) Checked the "Use Cursor" option (with a setting of 10,000) on the largest database input table (which was the MAIN row into the tMap)
Thanks.
