I'm doing a simple, three component job - tFileInputExcel > tMap > MSSqBulkOutputExec.
The input file has just 11,923 rows (was just written by another TOSDI job) and the tMap has no processing except some row mapping.
The MSSqBulkOutputExec uses a just-retrieved repository definition for the database table and is set to append to the SQL table.
Execution starts out slow and slows to a crawl until it stops at row 5,287 with the following: (The temporary file, mssql_data.txt finishes with 5,206 rows written.)
Exception in thread "main" java.lang.Error: java.lang.OutOfMemoryError: GC overhead limit exceeded
at masterproviderdatabase.dhpl_all_insertlicenseesintompdproviders_0_1.DHPL_All_InsertLicenseesIntoMPDProviders.tFileInputExcel_1Process(DHPL_All_InsertLicenseesIntoMPDProviders.java:3830)
at masterproviderdatabase.dhpl_all_insertlicenseesintompdproviders_0_1.DHPL_All_InsertLicenseesIntoMPDProviders.runJobInTOS(DHPL_All_InsertLicenseesIntoMPDProviders.java:4012)
at masterproviderdatabase.dhpl_all_insertlicenseesintompdproviders_0_1.DHPL_All_InsertLicenseesIntoMPDProviders.main(DHPL_All_InsertLicenseesIntoMPDProviders.java:3877)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.apache.xmlbeans.impl.schema.SchemaTypeImpl.createUnattachedNode(SchemaTypeImpl.java:1859)
disconnected
at org.apache.xmlbeans.impl.schema.SchemaTypeImpl.createElementType(SchemaTypeImpl.java:1021)
at org.apache.xmlbeans.impl.values.XmlObjectBase.create_element_user(XmlObjectBase.java:893)
at org.apache.xmlbeans.impl.store.Xobj.getUser(Xobj.java:1657)
at org.apache.xmlbeans.impl.store.Xobj.find_element_user(Xobj.java:2062)
at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTCellImpl.getIs(Unknown Source)
at org.apache.poi.xssf.usermodel.XSSFCell.getRichStringCellValue(XSSFCell.java:269)
at org.apache.poi.xssf.usermodel.XSSFCell.getRichStringCellValue(XSSFCell.java:64)
at masterproviderdatabase.dhpl_all_insertlicenseesintompdproviders_0_1.DHPL_All_InsertLicenseesIntoMPDProviders.tFileInputExcel_1Process(DHPL_All_InsertLicenseesIntoMPDProviders.java:2457)
... 2 more
I've done much bigger SQL outputs that this. I tied restarting Windows(7) to clear any cobwebs but no difference.
Any suggestions? Thanks!
UPDATE: I just replaced the MSSqBulkOutputExec then the tMap with a tLogRow, disconnecting them until I only have the FileInputExcel and the tLogRow - it still fails the same! ??
UPDATE: On the Advanced Settings tab of the FileInputExcel I found the Generation Mode field and set it to "Less memory consumed . . . " All set now.