topic Re: tMap doesn't store on disk in Data Quality

tMap doesn't store on disk

Anonymous — Fri, 09 Mar 2012 11:43:52 GMT

Hi everyone,
I got the well known problem of the java OutOfMemoryError by using the tMap.

Exception in thread "main" java.lang.Error: java.lang.OutOfMemoryError: Java heap space
	at test.test2_0_1.test2.tFileInputDelimited_1Process(test2.java:2367)
	at test.test2_0_1.test2.runJobInTOS(test2.java:4213)
	at test.test2_0_1.test2.main(test2.java:4087)
Caused by: java.lang.OutOfMemoryError: Java heap space

I set a path in the "stock on disk" field but, during the running, no file is created into the directory. No writing rights restriction.
Thanks by advance

Re: tMap doesn't store on disk

_AnonymousUser — Mon, 12 Mar 2012 13:43:57 GMT

Hi,
What have you set as max number of elements on the advanced property of the tMap, have you tried with a really low value to see if Talend does use the disk ? It seems that in recent version (5.0.1+?) Talend doesn't crash on this anymore but just issues a warning

Warning: to avoid a Memory heap space error the buffer of the lookup has been limited to a size of 397827 , try to reduce the advanced parameter "Max buffer size" (~100000 or at least less than 397827), then if needed try to increase the JVM Xmx parameter.

maybe you should try upgrading if you havent' already ?
Hope that helps,

Re: tMap doesn't store on disk

Anonymous — Tue, 13 Mar 2012 13:22:58 GMT

Hi,
I'm using the version 4.2.2 because I need Talend Spatial and it doesn't seems to be available on v 5.x
I tried your test but it didn't work

Re: tMap doesn't store on disk

janhess — Tue, 13 Mar 2012 14:37:17 GMT

Have you selected the 'store temp data on disk' icon for the lookup in tMap?

Re: tMap doesn't store on disk

Anonymous — Tue, 13 Mar 2012 17:28:40 GMT

If you talk about the bouton near the field, yes I did.

Re: tMap doesn't store on disk

Anonymous — Thu, 22 Mar 2012 08:15:37 GMT

Have you selected the 'store temp data on disk' icon for the lookup in tMap?

Ok, I didn't see it, sorry and thanks. I Join an image for the guys as blink as me.
I have an other problem but it's with the spatial module. It appears that tMap doesn't import Geometry libraries beacause I get that readGeometry() and writeGeometry(Geometry) are undefined when I set "store temp data on disk" to true
any advice?
EDIT: I add I'm using Talend 5 with the Spatial module (4.2). This problem doesn't occur with Talend 4.2 but it is the only one I get.

Re: tMap doesn't store on disk

Anonymous — Thu, 19 Apr 2012 10:59:49 GMT

Hi guys,
I have the same problem as Patheu. When i want to "store on disk" i got this :
connecting to socket on port 3854
connected
Warning: to avoid a Memory heap space error the buffer of the lookup has been limited to a size of 1099050 , try to reduce the advanced parameter "Max buffer size" (~100000 or at least less than 1099050), then if needed try to increase the JVM Xmx parameter.
And i don't understand want can i do. I change the jvm parameters, nothing change. So i want to try the first proposal : "try to reduce the advanced parameter "Max buffer size"", but how to do?
i use : TOS Version: 5.0.2 Build id: r78327-20120216-0244 + Pc Windows xp pro sp3 3Go of RAM.
Thanks a lot,
Best regards,
Dex