I'm a new Qlik Sense business analysis and I'm building a Qlik Sense app for analysis to replace excel VBA code to pull historical purchases for one or more items - IT will no longer support the excel VBA :(. My organization is large and the total number of records in my PO data warehouse exceeds over well over 100M records. Requirement is that I use Qlik Sense to pull the same data as the excel VBA code.
My only search criteria is item number. There is no limit to time, geography or entity.
When i build the SQL to the data warehouse, I write the SQL in the Data Load Editor and it pulls all PO data... over 100M+ records. Couple of challenges:
1/ When Qlik Sense users attempt to use the app, they often run into Out of Memory errors.
2/ When the App works, it takes more than 20min for the sheet to display and users can enter item numbers.
Questions:
1/ is this expected behavior for Qlik Sense to frequently result in errors for such large data sets? I filtered the data sets to return about 50M records and it still results in error, sometimes.
2/ Is there a way in Qlik Sense to allow users to enter the item numbers BEFORE the initial data load and have only the required item number PO history be pulled?
3/ I tried building a QVD file but it takes 20 min to load the data then another 30 minutes to index. Waiting nearly an hour to run a report doesn't seem to make sense.
What other options, if any, do i have in Qlik Sense to limit the initial data load?
