java.lang.OutOfMemoryError: Java heap space

[Anonymous]

Hi all,

I'm new to the Talend Cloud Real-Time Big Data Platform.

We have the following scenario: a connection to a remote Azure Data Lake, where we try to write a file of 600MB of size. 

The component we use is a tFileInputRaw. When the job starts, after a couple of seconds it generates the following error: "java.lang.OutOfMemoryError: Java heap space". 

Of course it is a memory related issue. But changing the JVM settings didn't help.

As far as I can see the component itself has an option that says "Stream the file" and I feel like that this option would solve the issue. 

Is there anyone who implemented it? The documentation looks incomplete. 

Also other suggestions are welcome. 

[Anonymous · Accepted solution]

Hello,

When you are streaming a file from a component like tFileInputRaw, it is intended to load the whole file into memory.

In other words, the full raw component reads the full raw file and put it into memory.

Best regards

Sabrina

[Anonymous]

Hi Sabrina,

thank you! Exactly, the full raw component is what we used and solved the issue! 

[Anonymous]

Hello,

Great it works. Thanks for marking this topic as resolved.

Best regards

Sabrina