Dear community,
I am reading Excel files with the tFileInputExcel, tMap and tJavaRow components. If the Excel files get too large, I get an Out of Memory Error concerning the Java Heap Space. I've read in several forums and internet pages that in this case I can use a Lookup table in my tMap component, where I can set the property "Store on disk" to true and also the path, where the input data will be saved on the disk. To make this work I added a second tFileInputExcel setting the Lookup link to the tMap, which I use to map all incoming rows to the output table. The initial tFileInputExcel with the Main link to the tMap now hasn't got any mapping columns to the output table anymore. When I start the job, I still get the Java Heap Space error. Did I forget something? Or is there an example somewhere how to design the job if I only need the Lookup table for this purpose?
Kind regards
Sylvia
Probably the problem is reading the large Excelfile. For the main flow you can try the component tFileExcelWorkbookOpen (with memory saving mode if you use xlsx files) and tFileExcelSheetInput (and here read only the columns you need).
This way I guess you can decrease the memory usage significant.
tFileExcelWorkbookOpen --OnComponentOK--> tFileExcelSheetInput_1 -- main flow --> tMap_1
tFileExcelSheetInput_2 -- lookup flow --> tMap_1
If you read for lookup the same excel file, you need only one tFileExcelWorkbookOpen.
Hi jlolling, I could not find the tFileExcelWorkbookOpen component in TOS 5.3.1. Guess it might be available only in enterprise edition..
Is there anything else I can do to make tMap work with files that have high number of rows..?
