Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi there, looking for help ASAP. I'm getting this error when trying to dedup a HUGE text pipe delimited file. It is about 120GB. When I get this error, I usually play with the JVM arguments and I get it to run, but in this case I am not able to remedy it so far with changing the JVM arguments. It processes to about the 50M row level and it has over 100 million rows.
The latest arguments I had were this:
https://www.screencast.com/t/Sr0GU4B8kq9
And this is my workflow which is pretty simple. I just need to dedup the GetCurrentFile.
https://www.screencast.com/t/YZ7wYsI8BXJ
Any thoughts?
Thank you
Hello,
Could you please show us the error full stack trace?
Please feel free to let us know if these related articles help.
https://community.talend.com/t5/Migration-Configuration-and/OutOfMemory-Exception/ta-p/21669
Best regards
Sabrina