topic Re: csv file to redshift in Talend Studio

csv file to redshift

Anonymous — Sat, 16 Nov 2024 11:23:37 GMT

Hi,
I am trying to load data from csv file to redshift.
Tfileinputdelimited >> tmap >> tredshiftoutput.
But, I am getting these errors:
Starting job red_mysqltest at 17:19 09/12/2014.
connecting to socket on port 3890
connected
Exception in component tRedshiftOutput_1
org.postgresql.util.PSQLException: An I/O error occured while sending to the backend.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:283)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:332)
at project_1.red_mysqltest_0_1.red_mysqltest.tFileInputDelimited_1Process(red_mysqltest.java:1449)
at project_1.red_mysqltest_0_1.red_mysqltest.runJobInTOS(red_mysqltest.java:2247)
at project_1.red_mysqltest_0_1.red_mysqltest.main(red_mysqltest.java:2091)
Caused by: java.io.EOFException
at org.postgresql.core.PGStream.ReceiveChar(PGStream.java:276)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1654)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
... 6 more
disconnected
Job red_mysqltest ended at 17:47 09/12/2014.
Please tell me why these errors are coming??
Regards,
Vinuta

Re: csv file to redshift

Anonymous — Wed, 10 Dec 2014 10:22:06 GMT

Hi,
Have you already checked if there is something wrong with network bandwidth? And also check if there is some restriction on data transmission over certain limit(Firewall issue)?
Best regards
Sabrina

Re: csv file to redshift

Anonymous — Wed, 10 Dec 2014 10:36:39 GMT

Hi,
My internet connection has some problem.It keeps on disconnect every half an hour.But sometimes Internet connection will be proper.But, still I get like this error. Is that error because of Internet connection???
I do not know about firewall issue??
Can you please explain me more about that??How to fix it??
I am Getting error as for input string : "http". How to resolve this???
Regards,
Vinuta

Re: csv file to redshift

Anonymous — Wed, 10 Dec 2014 13:41:28 GMT

Hello,
While try to load data from csv file to redshift I got these errors:
Starting job red_mysqltest at 13:44 10/12/2014.
connecting to socket on port 3430
connected
Exception in component tRedshiftOutput_1
org.postgresql.util.PSQLException: ERROR: /rds/bin/padb.1.0.867/data/exec/58/0: failed to map segment from shared object: Cannot allocate memory
Detail:
-----------------------------------------------
error: /rds/bin/padb.1.0.867/data/exec/58/0: failed to map segment from shared object: Cannot allocate memory
code: 1015
context: dlopen(/rds/bin/padb.1.0.867/data/exec/58/0,RTLD_LAZY)
query: 4234372
location: exec_plan.cpp:2213
process: padbmaster
-----------------------------------------------
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2096)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1829)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:332)
at project_1.red_mysqltest_0_1.red_mysqltest.tFileInputDelimited_1Process(red_mysqltest.java:1056)
at project_1.red_mysqltest_0_1.red_mysqltest.runJobInTOS(red_mysqltest.java:1802)
at project_1.red_mysqltest_0_1.red_mysqltest.main(red_mysqltest.java:1646)
disconnected
Job red_mysqltest ended at 14:14 10/12/2014.
Please tell me the solution.
Regards,
Vinuta

Re: csv file to redshift

Anonymous — Thu, 11 Dec 2014 10:29:24 GMT

Hi,
We cannot get this issue before. On which official version did you get that? What's your OS? Is the redshift DB connection successful in DB metadata?
Can you connect to your Redshift DB successfully by client without using Talend Tool?
Best regards
Sabrina

Re: csv file to redshift

Anonymous — Thu, 11 Dec 2014 10:52:03 GMT

Hello,
Talend version:5.6.
OS: Windows 8.1
Job design:
tfileinputdelimited >> tmap >> tredshiftoutput
Yes. Redshift connection is successful in Meatdata.
Yes. I can connect to my redshift DB successfully without using Talend tool.
I am also getting for input string 'http' error.
I have selected the option trim columns in tfileinputdelimited component. But its not working.
Please help me to resolve the both errors.
Regards,
Vinuta

Re: csv file to redshift

Anonymous — Fri, 12 Dec 2014 09:31:59 GMT

Hi,
How did you set your tRedShiftOutput component? What's the rows per insert(commit every)? Any data transmission restrictions/limits in your DB? Could you please show us more job setting screenshots?

Best regards
Sabrina

Re: csv file to redshift

Anonymous — Fri, 12 Dec 2014 12:16:48 GMT

Hello,
commit size: 100
There is no data transmission limits.

Can we use any other component instead of tredshiftoutput component ???
Regards,
Vinuta

Re: csv file to redshift

Anonymous — Mon, 15 Dec 2014 08:20:00 GMT

Hi,
Have you tried to check out "Die on error" on tRedShiftOutput component to see if there is any error info printed on console?
Best regards
Sabrina

Re: csv file to redshift

Anonymous — Mon, 15 Dec 2014 08:54:40 GMT

Hi,
I have not select the box die on error.
Can we use other component to load data?? CAn you please tell me??
Regards,
Vinuta

Re: csv file to redshift

Anonymous — Mon, 15 Dec 2014 09:28:30 GMT

Hi,
Can you use tRedShiftRow to insert data into Redshift DB?
Best regards
Sabrina

Re: csv file to redshift

Anonymous — Mon, 15 Dec 2014 10:14:56 GMT

Hi,
Can you please explain me more about tredshiftrow??
How to load the data into tredshiftrow??