Error: java.lang.OutOfMemoryError: Java heap space

[Anonymous]

Hi everybody.

Again, I REALLY need your help

I'm having the error "Exception in thread "main" java.lang.OutOfMemoryError: Java heap space".

My OS is Windows 7 Professional Service Pack 1, 64 bits and my RAM 8GB. I'm using Talend Open Studio for Data Integration, Version: 7.0.1

 

Right now, I'm trying to read an Excel file with 1 million rows and 34 columns aprox. full of data, using tFileInputExcel and a tLogRow, but my job only reads the first row (header) and then I get the error.

 

If I can read all the data (I hope you can help me with this), I'll process the information with components such as tMap, tAggregateRow, tPivotToColumnsDelimited, tFilterRow, tHashInput and tHashOutput, sending the entire results to a tFileOutputExcel & tFileOutputDelimited.

  My advanced settings are:

 -Xsm256M

 -Xmx2048m

 -XX:-UseGCOverheadLimit

Any suggestion? If somebody thinks that I could send you the file I'm trying to read (to ckeck if the problem is my computer), just tell me (it's an excel file .xlsx, 153MB)

Thanks!

[akumar2301]

Hello ,

I am not able to replicate.I see output same as it is in excel. It reads date with pattern "dd-MM-yyyy hh:mm:ss" also as a string.

Can you attach one sample input?

[Anonymous]

Hi @uganesh 

I have attached a very small sample of the input file and how it looks once it passes through column and row filters (which I require in the job). The interesting thing as I mentioned before is the change of the content in the column 'EventTime' that corresponds to a date (select one of the cells in the input file so you can see how it is inside, not only in the preliminary view).

Let me know if you can notice in the output file what I mean about the change when I have to configure the column as a string.

Thanks!

Output.xlsx
Input.xlsx

[akumar2301]

It is because of custom formatting of that excel column. By Any chance , could you request to change the custom format in INPUT to dd/MM/yyyy hh:mm:ss" ?

[Anonymous]

Hi @uganesh 

I had already tried, but I get this error:

Using this scheme (it's important that you know that using the User Mode, I had not had any error with this scheme):

And previously setting this format to that column in the excel file:

what do you think?

Thanks!

[akumar2301]

with event mode , in schema date pattern put
dd/MM/yyyy HH:mm

excel custom pattern dd/mm/yyyy hh:mm ( as highlighted above)

it works for me.you date pattern (excel and Java) should be compatible.

[Anonymous]

@uganeshI just did it (date pattern in Talend like "dd/MM/yyyy HH:mm", I tried it with quotation marks and without quotation marks, with the month and the hour in uppercase and in lowercase) and I'm getting the same error.

I know almost nothing about Java, but could it be that the JRE installed in Talend says it is 1.8.0_171 and in Java I have 1.8.0_191? And if it is this, what should I change?

[akumar2301]

@MayTorres attaching simple job which reads your sample input excel file in String and Date format ( after Custom format change )

readEXCEL_0.1.zip
Input.xlsx

[Anonymous]

Well @uganesh, this is embarrassing because your help has been amazing, but I tried your job with the file that I'm attaching here (after custom format change) and I still get the error. I really do not know what I can be doing wrong

Update: I just used the excel file 'Input' that you loaded with the job, and the same error comes out. I think definitely the problem is not the file but some unwanted behavior between Talend and Java when using the Event Mode.

Input.xlsx

[David_Beaty]

Hi,

Open the job in STudio and click the "Code" tab at the bottom of the screen. You should see a red section highlighted where the right hand side slider bar is showing you where the problem is.

Thanks