Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
Tian-Jay
Contributor II
Contributor II

Data load deletion

So i have an app that loads a huge amount of data and also performs a tonne of calculation post load (e.g. in the master measures) i am trying to improve the efficiency of the app and clean it up a little. To improve loading efficiency i can move some of the calculations to the qlik data load editor but when i do this i want to remove all the outdated information. 

Some of these Data sets have 8 million+ lines and often 75%+ of that is data that i cannot or should not use for analysis, how do i delete this data using the data load editor, so that i don't have to use so many conditional statements in the app itself.

e.g. A lot of the data points will have empty titles (unusable) or be dated back to 2013 (unsuable) so filtering by name or by dates etc

Labels (2)
1 Reply
rwunderlich
Partner Ambassador/MVP
Partner Ambassador/MVP

You can add a where clause to your loadd statements to include only the rows you want. Something like

LOAD
 ...
From ...
Where Year(OrderDate) >= 2020 and len(Title) > 0;

-Rob