Skip to main content
Announcements
SYSTEM MAINTENANCE: Thurs., Sept. 19, 1 AM ET, Platform will be unavailable for approx. 60 minutes.
cancel
Showing results for 
Search instead for 
Did you mean: 
Tian-Jay
Contributor II
Contributor II

Data load deletion

So i have an app that loads a huge amount of data and also performs a tonne of calculation post load (e.g. in the master measures) i am trying to improve the efficiency of the app and clean it up a little. To improve loading efficiency i can move some of the calculations to the qlik data load editor but when i do this i want to remove all the outdated information. 

Some of these Data sets have 8 million+ lines and often 75%+ of that is data that i cannot or should not use for analysis, how do i delete this data using the data load editor, so that i don't have to use so many conditional statements in the app itself.

e.g. A lot of the data points will have empty titles (unusable) or be dated back to 2013 (unsuable) so filtering by name or by dates etc

Labels (2)
1 Reply
rwunderlich
Partner Ambassador/MVP
Partner Ambassador/MVP

You can add a where clause to your loadd statements to include only the rows you want. Something like

LOAD
 ...
From ...
Where Year(OrderDate) >= 2020 and len(Title) > 0;

-Rob