topic Re: tHashOutput and tHashInput Capacity in Talend Studio

tHashOutput and tHashInput Capacity

Anonymous — Thu, 21 Apr 2016 21:39:22 GMT

Hi Team,

I am trying to load and read 30 million records in tHashxx components. The records will look like (key, value1, value2) . I need to reload and read tHashxx for every iteration till 30 iterations.

I am able to read till 280 thousands using tMap but not more than that. Please suggest on how to achieve this?

Attached is the screenshot of the job flow for your reference.

tHashOutput3 and tHashInput2 will have the 30million records.

Note : as of now I am limiting the data coming from the tDB2Input.

Re: tHashOutput and tHashInput Capacity

Anonymous — Mon, 25 Apr 2016 11:05:41 GMT

Hi,
Did you get any outofmemory or heap space error? or just job is hanging ?
What's the error you are getting when read thousands data by using tMap?
Best regards
Sabrina

Re: tHashOutput and tHashInput Capacity

Anonymous — Mon, 25 Apr 2016 14:23:36 GMT

Hi, am getting out of memory issue. My RAM is 16 gig.

Re: tHashOutput and tHashInput Capacity

Anonymous — Mon, 25 Apr 2016 14:46:30 GMT

Hi,
You can raise your JVM

Go to the job execution panel

Thanks,

Amine

Re: tHashOutput and tHashInput Capacity

Anonymous — Tue, 26 Apr 2016 16:53:40 GMT

Hi, I have raised the JVM values, limited the number of columns read but still am unable to achieve.

Re: tHashOutput and tHashInput Capacity

Anonymous — Wed, 27 Apr 2016 15:01:10 GMT

Hi,
Could you please post the error message also your JVM settings!
thanks