tFileInputExcel with more than approx. 2500 rows gives stackoverflow
Does anyone else have problems with excel as input? It seems to be the Pattern (regex) which gives a stackoverflow when there is a bit more than 2500 rows. I have 7000 rows, but the error always comes after approx. 2500 rows have been processed. Exporting to CSV and doing the same parsing gives me no problems. As a sidenote, it seems much faster to process CSV.
Hi,
Thank you a comprehensive testing and summary.
Available when Read excel2007 file format (xlsx) is selected in the Basic settings view. Select the mode used to read the Excel 2007 file.
Generation mode:
Less memory consumed for large excel(Event mode): used for large file. This is a memory-saving mode to read the Excel 2007 file as a flow.
Memory-consuming (User mode): used for small file. It needs much memory. That is the reason
Now I tried to change the advanced setting for "Generation mode" from "Memory-consuming", to "Less memory consumed". That will also do the trick, and actually "Less memory consumed" reads all sheets faster than when using "Memory-consuming" So I guess that formulas in the sheet and "Memory-consuming" do not work so well together. And based on tests, it seems that "Less memory consumed" is faster anyway.
For more details, see the component reference
tFileInputExcel Best regards
Sabrina
Memory-consuming (User mode): used for small file. It needs much memory. That is the reason
Yes, I read that. But that documentation does not propose any benefit of using that mode. I don't think that "It needs much memory" qualifies as a benefit. And what is a "small" file anyway?
Based on my small test, I see no benefit in that "small but memory-consuming" mode at all. In my test, it is not any faster (which would be a benefit).
My suggestion to remove it was more like a suggestion for an easy fix and less clutter at the same time. But I will leave that up to the Talend team.
(I apologise for mixing in a comment about xml. That DOES have some special purpose, although I have yet to finde examples taking advantage of it)
Hi Jsi,
Could you open an issue for the Doc team, in
https://jira.talendforge.org/browse/DOCT, please?
We need to be more accurate in the documentation, it seems, and we also need to clarify with the dev team in which cases it is interesting to use the more "memory-consuming" mode. We missed that point, it seems.
Cheers,
Elisa
