I am creating an application in Qlikview 8.5 that analyses data from log files. The log files are about 4 Mb from each machine and I want to analyze as many machines as possible.
The problem I experience is that I very often get out-of-memory problems, mostly when loading but often the program crashes when I managed to load the data but it cannot publish a graph.
Despite having worked with Qlikview for a while now I am still a newbie. I am sure my code is far from optimal.
I need help with how to prioritize my memory trouble shooting so I don't waste too much time for miniscule gains.
What I would like is a prioritized list of what I should try to work for.For example.
1. Change count() to sum() where possible
2. Join tables that could be joined (if this would be a good idea)
5. prepare data in a script if it possible, this will allow you to create simpler expressions
6. tables should be associated by fields that hold numbers not string. If your join field is a text field than change it using autonumber, autonumberhash, etc.
I would organize my loading process into those steps (if you have one log file per day) - Loader.qvw: 1. load old data from existing qvd (Data.qvd) (on the first load you want have any qvd so go to step 2) 2. append new data from txt file (we are adding new data), do some data manipulations if it is necessary 3. store data to Data.qvd. Each day you append data from the previous day to your existing QVD (if you run the loader on the daily basis...). Then the application loads data from QVD.
Qlikview works faster when "join fields" are numeric.
If you have text field and you do count(distinct TextField) it is better to create a new field in the script and use sum(): load *, Textfield, 1 as Counter resdient table; Now you can use: sum(Counter). Important: in this example we assume that we have one distinct Textfield value per row.
