Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
vishavb007
Partner - Contributor III
Partner - Contributor III

Qlik Data Catalog Load data from S3

Hi All,

I am able to load data(data from single file) from S3 bucket into Qlik Catalog.

But I need to load the folder from S3 in which we have multiple files , kindly let me know if it is possible to load the folder & if it is yes , please let me know how to do it.

 

Kindly let me know if any further details are required regarding the use case.

 

Regards,

Vishav Bhatia

Labels (1)
1 Solution

Accepted Solutions
Hordy_Beauzile
Support
Support

Hi @vishavb007,

After adding the source connection of the S3 bucket, you should be able to select the folder as a source. After creating the source you should then be able to load the files through the established source connection.

https://help.qlik.com/en-US/catalog/May2022/Content/QlikCatalog/Source/JDBC_New_Source.htm

Best,

Hordy

View solution in original post

2 Replies
Hordy_Beauzile
Support
Support

Hi @vishavb007,

After adding the source connection of the S3 bucket, you should be able to select the folder as a source. After creating the source you should then be able to load the files through the established source connection.

https://help.qlik.com/en-US/catalog/May2022/Content/QlikCatalog/Source/JDBC_New_Source.htm

Best,

Hordy

ducetcro
Contributor
Contributor

Split the files into 100Mbs chunks and convert to parquet (for best read performance). Partition them by Day or by Hour (to reduce cost when querying). Create a Redshift Spectrum external table pointing to the bucket and access the files directly without needing to move them.