Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi Qlik Team,
I'm curious about the writing method Qlik Replicate uses in Big Query. Can you provide some insights?
Thanks,
Harsh
Hello @harsh2 ,
Thanks for reaching to Qlik Community!
By default Qlik Replicate utilize the interim CSV files to LOAD the data (in Full Load stage, it's history data records, in CDC stage it's change data records) to Google BigQuery rather than INSERT the data rows straightly to gain better performance. Qlik Replicate composes the .CSV files (at Qlik Replicate Server) and then call LOAD command, upload the .CSV files to BigQuery storage then merge the data from interim CSV files to final target table(s).
If you like you may set TARGET_LOAD & TARGET_APPLY to Verbose or Trace to get more information to understand the behavior further.
Hope this helps.
John.
Hello @harsh2 ,
Thanks for reaching to Qlik Community!
By default Qlik Replicate utilize the interim CSV files to LOAD the data (in Full Load stage, it's history data records, in CDC stage it's change data records) to Google BigQuery rather than INSERT the data rows straightly to gain better performance. Qlik Replicate composes the .CSV files (at Qlik Replicate Server) and then call LOAD command, upload the .CSV files to BigQuery storage then merge the data from interim CSV files to final target table(s).
If you like you may set TARGET_LOAD & TARGET_APPLY to Verbose or Trace to get more information to understand the behavior further.
Hope this helps.
John.