Skip to main content

Suggest an Idea

Vote for your favorite Qlik product ideas and add your own suggestions.

Announcements
Action-Packed Learning Awaits! QlikWorld 2023. April 17 - 20 in Las Vegas: REGISTER NOW

Qlik Replicate - AWS S3 as a target support for Parquet file format.

Deepak1
Contributor III
Contributor III

Qlik Replicate - AWS S3 as a target support for Parquet file format.

enable support for parquet for AWS S3 target.
Enabling support for parquet will help provide better solution for datalake users. Apache Parquet is open source file format. Parquet is designed for efficient as well as performant flat columnar storage format for data as compared to csv files. Parquet works much bette rand efficient with complex data in bulk and features efficient data compression and ecoding.

Please find a sample comparision between Parquet and CSV with AWS S3 in terms of savings and speed converting data in Parquet and CSV:

DatasetAWS S3 SizeQuery TimeData ScannedCost ($)
Data stored as CSV1 TB250 seconds1.15 TB$6
Data stored as Parquet130 GB8 seconds2.72 GB0.03
Savings87% less with parquet34 times faster99% less data
scanning
99% more
savings
1 Comment
Shelley_Brennan
Employee
Employee

Thank you for the suggestion.  We do have a similar Ideation post here that you can follow and be kept up to date on current status of support for Parquet file format for AWS S3 targets.  

https://community.qlik.com/t5/Suggest-an-Idea/Qlik-Replicate-parquet-output-format-for-S3-endpoint/i...

Status changed to: Closed - Duplicate