Skip to main content
Announcements
Introducing Qlik Answers: A plug-and-play, Generative AI powered RAG solution. READ ALL ABOUT IT!
cancel
Showing results for 
Search instead for 
Did you mean: 
teach_me
Contributor II
Contributor II

Loading large number of data from source

Hi,

I want to load large amount of data daily(1 Millions records) from source. please suggest me any steps. and how can i take care of performance issue.

Labels (1)
1 Reply
andrew_smith200
Contributor III
Contributor III

Very much depends on how many columns you have. I daily load in 37 Million, 50 column data table and additional reference tables - does the lot in 3-4 minutes. However that is working directly from .qvd files.

My recommendation would be to build and app which builds qvds then load them into your app. You would need to decide how you build those qvds - incremental, insert, update or recreate and then how you would divide them up as necessary - fr example by month or quarter etc.