Skip to main content

Suggest an Idea

Vote for your favorite Qlik product ideas and add your own suggestions.

Announcements
July 15, NEW Customer Portal: Initial launch will improve how you submit Support Cases. IMPORTANT DETAILS

Qlik Replicate - AWS S3 as a target support for Parquet file format.

Deepak1
Contributor III
Contributor III

Qlik Replicate - AWS S3 as a target support for Parquet file format.

enable support for parquet for AWS S3 target.
Enabling support for parquet will help provide better solution for datalake users. Apache Parquet is open source file format. Parquet is designed for efficient as well as performant flat columnar storage format for data as compared to csv files. Parquet works much bette rand efficient with complex data in bulk and features efficient data compression and ecoding.

Please find a sample comparision between Parquet and CSV with AWS S3 in terms of savings and speed converting data in Parquet and CSV:

DatasetAWS S3 SizeQuery TimeData ScannedCost ($)
Data stored as CSV1 TB250 seconds1.15 TB$6
Data stored as Parquet130 GB8 seconds2.72 GB0.03
Savings87% less with parquet34 times faster99% less data
scanning
99% more
savings
1 Comment
Shelley_Brennan
Former Employee
Former Employee

Thank you for the suggestion.  We do have a similar Ideation post here that you can follow and be kept up to date on current status of support for Parquet file format for AWS S3 targets.  

https://community.qlik.com/t5/Suggest-an-Idea/Qlik-Replicate-parquet-output-format-for-S3-endpoint/i...

Status changed to: Closed - Duplicate