topic Netezza External Tables in Talend Studio

Netezza External Tables

Anonymous — Thu, 12 Jul 2012 21:51:44 GMT

Hi
I am trying to extract data from Netezza DB using external table concept using tNetezzaRow/tNetezzaInput component and it is giving Out of space exception after 1 Gb of extract or it will extract only 1 Gb of data and job still be in RUN status.
So, I had to use tNetezzaInput component and Dynamic Datatype(The reason to use Dynamic datatype is to extract any table) to extract data. But that is very slow. Can some one help me what I am doing wrong and where I can make changes to make external table concept work? If not External table is there any way i can extract data much faster. Now I am getting output like 1Gb per Hr.
I am using Talend 4.1.1
Code:
"CREATE EXTERNAL TABLE 'xyz.dat'
USING
(
DELIMITER ','
QUOTEDVALUE 'DOUBLE'
Y2BASE 2000
ENCODING 'internal'
REMOTESOURCE 'JDBC'
ESCAPECHAR '\'
)
AS
SELECT *
FROM xyz;"

Re: Netezza External Tables

Anonymous — Fri, 13 Jul 2012 02:24:54 GMT

Hi
You mean outofspace or OutOfMemory exception?
Please show us the sceen shot of your job. Then we may know where to optimize it.
Regards,
Pedro

Re: Netezza External Tables

Anonymous — Fri, 13 Jul 2012 13:13:26 GMT

I am sorry. Yes, OutOfMemory exception.
I attached my Talend snap shot.

Re: Netezza External Tables

Anonymous — Fri, 13 Jul 2012 13:16:27 GMT

Here is the snap shot.

Re: Netezza External Tables

Anonymous — Mon, 16 Jul 2012 02:32:28 GMT

Hi
Please increase the JVM arguements as the following image.
Regarda,
Pedro

Re: Netezza External Tables

Anonymous — Tue, 17 Jul 2012 13:50:16 GMT

HI Pedro
I did increase the JVM arguments and got the error. If I take it as 1024 it will extract data of 1 Gb and then keeps quiet and then I increased the size and it either craps out with an error "OutOfMemory" exception or it will extract 1 Gb data and does nothing(it neither dies or increase the file struncture)

Re: Netezza External Tables

Anonymous — Wed, 18 Jul 2012 03:26:48 GMT

Hi
I try to find out some components in your job which can support "store on disk" feature. But none support it.
So increasing JVM arguments is an effective way.
Besides, please select the option Advanced Settings->Use cursor on tNetezzaInput.
Regards,
Pedro