Suggest an Idea

[Prabodh]

We would like to use File target endpoint to populate our datalake using Replicate. However, our datalake file format is Avro which is not currently supported by Replicate.

Please add support for Avro and Parquet file formats in File target endpoint.

[simonaubert]

Hello@Prabodh 

May be a stupid question, but why don't you use Orc instead of Avro?

It's fastest and has better compression. Also, it works great with hive or druid.

Best regards,

Simon

[Shelley_Brennan]

Thank you for the feedback.  Do you have any more details on your use case and if you've considered using Qlik Compose to build out the data lake, which does support Avro and Parquet file formats?

[Prabodh]

@simonaubert Avro is our standard data lake format.

@Shelley_Brennan Compose would not work for us. We need Replicate to land files in Avro format. Also, to be specific, we need the capability to write Avro files for S3 endpoint on priority.

[Shelley_Brennan]

Support for additional file formats is in our roadmap.  We will update when development is underway.  Please continue to share any additional feedback here in the meantime.  Thank you!

[Nulee_Massaro]

Partially delivered - The option to create files in Parquet format has been added to the Amazon S3 target endpoint.