Query s3 data using Redshift Spectrum

[desanip]

Hello Talendians,

I have a use case where i have to query directly from S3 using Redshift Spectrum. The data stored in s3 is in parquet format. We already have external schema and tables created using Glue Catalog. What should be the design approach? I see Talend does not have any Spectrum Components. Only RedshiftBulkexec and redshiftoutput components. Can someone throw somelight on how we can achieve this in Talend? 

[Anonymous · Accepted solution]

Hi,

    I have not personally tried connecting Redshift Spectrum using Talend Redshift components. Are you getting any error while connecting using them? As a fallback strategy you can check generic JDBC DB components also to connect the Spectrum.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

[desanip]

Hi Nikhil,

Thanks for the reply

I don't think there are any Spectrum Components in Talend. Like you said i can use generic Database connection to connect to spectrum. I was trying to do a proof of concept to query data from s3 using Redshift spectrum and was wondering how do it.