Regarding External Tables and Temporary Tables with Azure Synapse Analytics as the Target

[iti-attunity-sup]

Hi, all.

In Qlik Replicate, when targeting Azure Synapse Analytics, the data processing flow appears to operate as follows:

• Uploading CSV files to storage

• Creating an external table based on the CSV files

• Performing a COPY INTO operation from the external table into a temporary table

• Joining the temporary table with the target table to execute a bulk UPDATE/DELETE

Regarding this workflow, I have a question about the design: If we were to join the external table directly with the target table instead of using a temporary table, we could skip the COPY INTO process entirely. I assume this would be faster and more efficient.

Could you please explain the reason why Qlik Replicate is implemented to follow this specific flow?

Best Regards.

[Rahul_Kale · Accepted solution]

Hello iti-attunity-sup,

Thank you for reaching out to the Qlik community.
 

Qlik Replicate uses external table → COPY INTO → temp table → MERGE because:

• External tables are slow for joins and DML in Synapse. 
• Microsoft recommends loading into staging/temp tables first for best MPP performance. 
• COPY INTO is optimized for high‑throughput ingestion into native tables.
• Temp tables enable fast, set‑based CDC updates/deletes instead of row‑by‑row operations. 

Joining the external table directly would usually be slower and less scalable.

[iti-attunity-sup]

Hi, Rahul_Kale-san.

I now understand that external tables have a disadvantage in terms of speed, and that using temporary tables resolves this issue.
Thank you again for providing this very important information.

Best Regards.

[Rahul_Kale]

Happy to help @iti-attunity-sup.