topic Re: Load multiple excel files into Redshift in Talend Studio

Load multiple excel files into Redshift

Parikhharshal — Sat, 16 Nov 2024 03:47:52 GMT

Hi experts

Hope you are well.

I have got the scenario where I am looping in all 3 excel files stored locally and then loading them into Redshift.

I have designed my job like this:

This is my setup in tfileinputexcel.

It actually shows 2 exection finished but it actually loads only one file in redshift. Not sure what am I doing wrong.

Please help.

Thanks in advance.

Thanks

Harshal.

Re: Load multiple excel files into Redshift

Anonymous — Wed, 18 Dec 2019 03:15:32 GMT

Try to make a testing, add a tLogRow after tMap to the print the data on the console, to check if each file has data and all data are read from the file.

Re: Load multiple excel files into Redshift

Parikhharshal — Wed, 18 Dec 2019 03:23:23 GMT

Hi @shong

I added tlogrow and it works fine in there. Not sure what's the problem with tdbout component.

Re: Load multiple excel files into Redshift

Anonymous — Wed, 18 Dec 2019 03:49:42 GMT

Do you create the table on tRedshiftOutput? What is the option in Action on table list?

Re: Load multiple excel files into Redshift

Anonymous — Wed, 18 Dec 2019 14:26:19 GMT

Hi,

Could you please add a tRedshiftCommit to tRedshiftOutput using On Component Ok? Please do not close the connection since you need to repeat the iterations. You can close the connection in tPostJob section once the entire job is complete.

At the same time, add Die on error option in tRedshiftOutput component to make sure that data is indeed getting loaded without any error.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: Load multiple excel files into Redshift

Parikhharshal — Wed, 18 Dec 2019 22:47:20 GMT

Hi Nikhil and all

Thanks a lot for your reply.

I ended up resolving this by using existing connection instead of defining as new.

Thanks

Harshal.