topic Re: data is not getting loaded to Redshift database in Talend Studio

data is not getting loaded to Redshift database

sushantk19 — Sat, 16 Nov 2024 02:40:07 GMT

Hi,

I got a strange issue. I have created a simple job to load the data to Redshift Target Table. When i run this job, in Logs it shows me that is getting inserted, But actually the data is NOT getting inserted in Database. Is there any issue with JOB settings?

I checked the window after the job runs and it shows all the data correctly. This is the 1st time I am loading data in REDSHIFT tables after database has been setup. What could be possible reason?

Re: data is not getting loaded to Redshift database

manodwhb — Fri, 17 Apr 2020 00:58:14 GMT

Check below link and are you using same way?

https://help.talend.com/reader/OHwQvD9ynJK6fgSUqgOHKQ/r_wOvE2emirxIG1QvqXTxA

Re: data is not getting loaded to Redshift database

sushantk19 — Fri, 17 Apr 2020 09:59:28 GMT

now it is working as i reduced the commit every "rows" from Advanced settings. I was suggested to use the "Auto commit" option. But to do that, I have to use tRedshift connection component. I tried connecting that to my existing job, but it is throwing an error. Can you share if possible how to connect this tRedshift connectioncomponent with tRedshiftoutput component?

Re: data is not getting loaded to Redshift database

manodwhb — Fri, 17 Apr 2020 10:12:08 GMT

@sushantk19, check the below screenshot.

Re: data is not getting loaded to Redshift database

sushantk19 — Fri, 17 Apr 2020 10:15:37 GMT

ok. I have a different scenario. My source is like a SQL server and target is REDSHIFT table. Please see the screenshot attached.Please suggest where and how do i connect tRedshift connection component.

DB_connection.png

Re: data is not getting loaded to Redshift database

manodwhb — Fri, 17 Apr 2020 10:20:51 GMT

@sushantk19, before tMsSqlserverInput (tDBInput) component use tRedshiftconnection and connect using on component ok from tRedshiftconnection to tMsSqlserverInput (tDBInput) and in the tRedshiftOutput enable the check box of Use an existing connection.

Re: data is not getting loaded to Redshift database

sushantk19 — Fri, 17 Apr 2020 10:47:35 GMT

Works perfectly fine, but the performance is slower as compared to earlier run where commit interval was changed at the tRedshiftoutput component.

Re: data is not getting loaded to Redshift database

sushantk19 — Fri, 17 Apr 2020 11:43:27 GMT

Initially the job used to take ~5 mins to load 10000 records with commit every 10000 rows. Now with this component connected it has taken ~ 17 mins. Is there issue using this approach?