topic Re: Query s3 data using Redshift Spectrum in Talend Studio

Query s3 data using Redshift Spectrum

desanip — Sat, 16 Nov 2024 04:14:16 GMT

Hello Talendians,

I have a use case where i have to query directly from S3 using Redshift Spectrum. The data stored in s3 is in parquet format. We already have external schema and tables created using Glue Catalog. What should be the design approach? I see Talend does not have any Spectrum Components. Only RedshiftBulkexec and redshiftoutput components. Can someone throw somelight on how we can achieve this in Talend?

Re: Query s3 data using Redshift Spectrum

Anonymous — Tue, 29 Oct 2019 14:04:17 GMT

Hi,

I have not personally tried connecting Redshift Spectrum using Talend Redshift components. Are you getting any error while connecting using them? As a fallback strategy you can check generic JDBC DB components also to connect the Spectrum.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: Query s3 data using Redshift Spectrum

desanip — Wed, 30 Oct 2019 12:30:58 GMT

Hi Nikhil,

Thanks for the reply

I don't think there are any Spectrum Components in Talend. Like you said i can use generic Database connection to connect to spectrum. I was trying to do a proof of concept to query data from s3 using Redshift spectrum and was wondering how do it.