Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in NYC Sept 4th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
jjustingkm
Creator II
Creator II

load time databricks

I am trying to load a big table from databricks around 100 million records and is taking around 1.5 hours. 

Needed the detail data in the sheet. 

if we load the data into a QVD, would it make any difference in the load time ( I know that loading into a table and QVD both needs to hit the database).

Labels (1)
2 Replies
paulcalvet
Partner - Specialist
Partner - Specialist

Hello,

Yes the best solution in your case is to load from databricks in delta mode and store data into QVD.

You can split QVD.

Load from QVD is faster in Qlik and your records will be load in few minutes.

jjustingkm
Creator II
Creator II
Author

Thank you for your reply. I am basically looking for reduce the load from databricks. 

When you say delta mode, are you talking about incremental load. In this case there is no modification date in the table so incremental load is not possible.