idea Add parquet as a data format for Microsoft Azure Databricks endpoint in Suggest an Idea

Add parquet as a data format for Microsoft Azure Databricks endpoint

joelm777 — Thu, 27 Jan 2022 15:49:07 GMT

Please add parquet as a “target storage format”

Sequence may be faster than text, but it also cannot be easily maintained when the cdc table (__ct) is wanted for long term storage. Many small files are created and slow reads result when hitting the endpoint target tables. Parquet has more flexibility for maintenance and streaming. This can help performance and usability of the replicate endpoint, speeding up Qlik Compose views, etc.

Re: Add parquet as a data format for Microsoft Azure Databricks endpoint - Status changed to: Open - On Roadmap

TimGarrod — Wed, 02 Mar 2022 14:50:43 GMT

Thanks for the ideation.

This feature is on the roadmap for both Replicate DELTA (use parquet as the transient file structure) and for Replicate Databricks (Non-Delta) endpoint that is used with Compose.

Re: Add parquet as a data format for Microsoft Azure Databricks endpoint

mehmet_gencsoy — Fri, 03 Jun 2022 13:11:46 GMT

Any timeline for that feature?

Re: Add parquet as a data format for Microsoft Azure Databricks endpoint

JCJnr — Wed, 24 Aug 2022 10:47:13 GMT

Is this on the cards for the next release ? or when could we expect this?

From now on, please track this idea from the Ideation por...

Meghann_MacDonald — Wed, 02 Aug 2023 14:58:12 GMT

From now on, please track this idea from the Ideation portal.

Link to new idea

Meghann

NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.

Re: Add parquet as a data format for Microsoft Azure Databricks endpoint - Status changed to: Closed - Archived

Ideation — Wed, 02 Aug 2023 14:58:14 GMT